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EXECUTIVE  SUMMARY 

This  report  describes  in  detail  the  procedures,  methods,  and  resources  Parsons  used  to  complete 
the  demonstration  project  at  the  former  Camp  Beale  for  Environmental  Security  Technology 
Certification  Program  (ESTCP)  Munitions  Response  (MR)-201104  (Evaluation  and 
Discrimination  Technologies  and  Classification  Results)  and  ESTCP  MR-201157 
(Demonstration  of  MetalMapper  Static  Data  Acquisition  and  Data  Analysis).  The  2011  ESTCP 
Unexploded  Ordnance  (UXO)  Classification  Study,  Camp  Beale,  California,  was  conducted  with 
two  primary  objectives: 

•  Test  and  validate  detection  and  discrimination  capabilities  of  currently  available  and 
emerging  advanced  electromagnetic  induction  sensors  developed  specifically  for 
discrimination  on  real  sites  under  operational  conditions. 

•  Investigate  in  cooperation  with  regulators  and  program  managers  how  classification 
technologies  can  be  implemented  in  munitions  and  explosives  of  concern  (MEC)  cleanup 
operations. 

Parsons  had  two  separate  teams  working  on  the  project  under  two  different  ESTCP  project 
numbers,  ESTCP  021104  and  ESTCP  201157.  One  team  was  responsible  for  site  setup,  the 
placement  of  200  seed  items  for  use  in  measuring  the  capabilities  of  the  advanced  EMI  sensors 
tested,  the  subsequent  collection  of  the  9  acres  of  EM6-MK2  data,  and  the  intrusive  investigation 
of  the  2,143  targets  (including  seed  items)  selected  for  additional  investigation  with  the  advanced 
sensors.  The  second  team  was  responsible  for  the  cued  survey  of  1,491  of  the  2,143  targets  using 
the  MetalMapper,  one  of  the  advanced  electromagnetic  induction  (EMI)  sensors  tested  during 
this  project.  1,232  targets  were  surveyed  with  the  MetalMapper  in  the  open  field  area;  259 
targets  were  surveyed  with  MetalMapper  and  man-portable  sensors  in  the  combined  area;  and  the 
remaining  652  targets  were  selected  in  wooded  areas  for  the  testing  of  man-portable  sensors 
operated  by  other  demonstrators. 

The  MetalMapper  is  an  advanced  EMI  system  developed  by  Geometries,  Inc.,  with  support  from 
the  ESTCP.  It  has  three  mutually  orthogonal  transmit  loops  in  the  Z,  Y,  and  X  directions  and 
contains  seven  triaxial  receiver  antennas  inside  the  Z  (bottom)  loop,  allowing  21  independent 
measurements  of  the  transient  secondary  magnetic  field.  Data  were  collected  statically,  such  that 
one  data  point  was  collected  for  each  target  selected  for  investigation.  The  collected  data  were 
inverted  and  analyzed  using  the  UX-Analyze  add-on  to  Geosoft’s  Oasis  montaj  software.  Once 
analysis  was  complete,  a  theoretical  ranked  dig  list  (theoretical  because  all  targets  were 
intrusively  investigated  regardless  of  demonstrators’  stop-dig  points)  was  submitted  for  scoring 
by  the  Institute  for  Defense  Analysis  (IDA). 

Dig  list  scoring  was  based  on  the  number  of  targets  of  interest  (TOI)  correctly  identified  as  items 
that  should  be  dug  and  the  number  of  non-TOI  or  clutter  items  that  were  correctly  classified  as 
items  that  did  not  need  to  be  intrusively  investigated.  The  single  dig  list  submitted  by  Parsons 
was  scored  against  two  ground  truth  sets:  one  that  identified  a  number  of  native  fuzes  as  clutter, 
and  one  that  identified  these  fuzes  as  TOI.  Because  all  of  the  native  fuzes  were  officially 
classified  as  TOI,  the  fuzes  as  clutter  ground  truth  set  is  considered  the  more  descriptive  for 
MetalMapper  results.  Comparison  of  the  dig  list  with  this  ground  truth  set  resulted  in  the  correct 
identification  of  more  than  99  percent  of  the  TOI  on  site  (one  TOI  incorrectly  classified  as 
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clutter)  and  a  reduction  in  the  amount  of  clutter  that  would  have  been  dug  by  approximately 
78  percent. 

The  largest  implementation  issue  on  the  project  was  a  series  of  recurring  software  crashes  that 
led  to  an  average  production  rate  of  115  points  collected  per  day  with  the  MetalMapper. 
Although  the  exact  cause  of  the  crashes  is  still  unknown,  Geometries  is  aware  of  the  problem  and 
working  to  correct  it.  Additional  demonstration  studies  using  this  technology  are  planned  for 
this  year  and  upcoming  years,  and  it  seems  reasonable  to  assume  that  these  problems  can  be 
corrected  before  or  during  these  studies.  In  addition  to  potentially  refining  the  equipment,  future 
studies  look  to  focus  on  more  demanding  sites,  with  higher  anomaly  densities  and  a  wider  range 
of  munitions  sizes  than  those  at  Camp  Beale. 
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1.0  INTRODUCTION 


Currently  up  to  90  percent  of  excavation  costs  on  most  unexploded  ordnance  (UXO)  /  munitions 
and  explosives  of  concern  (MEC)  projects  are  related  to  removing  scrap  metal  that  does  not 
represent  an  explosive  hazard.  Significant  cost  savings  could  be  achieved  through  the  use  of 
geophysical  discrimination  methods  that  could  reduce  the  number  of  excavations  required  to 
remove  explosive  hazards  from  sites.  The  objective  of  this  project  is  to  demonstrate  the  use  of 
advanced  electromagnetic  induction  (EMI)  sensors  in  static  data  acquisition  mode  and  associated 
analysis  software.  To  achieve  these  objectives,  a  controlled  test  was  conducted  at  the  former 
Camp  Beale. 

1.1  BACKGROUND 

The  Fiscal  Year  2006  defense  appropriation  contained  funding  for  the  “Development  of 
Advanced,  Sophisticated  Discrimination  Technologies  for  Unexploded  Ordnance  (UXO) 
Cleanup.”  The  Environmental  Security  Technology  Certification  Program  (ESTCP)  responded 
by  conducting  a  UXO  discrimination  study  at  the  former  Camp  Sibert,  Alabama.  The  results  of 
this  first  demonstration  were  very  encouraging.  The  conditions  for  discrimination  were 
favorable  at  this  site  and  included  a  single  target  of  interest  (TOI;  4.2-inch  [in.]  mortar)  and 
benign  topography  and  geology.  All  of  the  classification  approaches  demonstrated  were 
correctly  identified  a  sizable  fraction  of  the  anomalies  as  arising  from  nonhazardous  items  that 
could  be  safely  left  in  the  ground.  Both  commercial  and  advanced  sensors  produced  very  good 
results.  Camp  San  Luis  Obispo  (SLO),  California,  was  the  site  for  the  second  study,  which 
provided  greater  challenges  in  topography  and  a  wider  mix  of  TOI.  Again,  the  results  were  very 
positive,  with  increased  discrimination  of  TOI  versus  nonhazardous  items.  In  2010,  the  third 
ESTCP  study  was  conducted  at  the  former  Camp  Butner,  North  Carolina,  which  all  included 
smaller  TOI  than  either  of  the  previous  sites.  Great  success  was  achieved  identifying  37- 
millimeter  (mm)  projectiles,  fuzes,  and  larger  TOI  with  the  advanced  sensors. 

To  build  upon  the  success  of  these  studies,  ESTCP  sponsored  a  fourth  study  in  2011  at  the 
former  Camp  Beale,  California,  a  site  with  a  wide  range  of  TOI,  moderate  to  steep  terrain,  and 
trees.  Past  studies  included  open  field  pastures;  the  current  study  area  included  four  acres  of 
medium  density  wooded  areas.  The  demonstration  area  at  the  former  Camp  Beale  included 
about  6  acres  of  open  field  area  and  3  acres  of  wooded  area  to  provide  increasing  difficulty  to 
test  the  high  standards  established  in  the  previous  studies.  Parsons  was  responsible  for  managing 
the  site,  performing  a  surface  sweep  of  the  survey  area,  seeding,  EM61-MK2  data 
collection/processing,  MetalMapper  data  collection/analysis,  and  excavating  the  1,943  anomalies 
associated  with  the  survey  area. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

This  type  of  approach  has  the  potential  to  reduce  the  number  of  excavations  required  to 
effectively  remove  the  explosive  safety  risk  (MEC)  at  a  given  site,  which  would  result  in 
significant  cost  savings  related  to  the  closure  of  formerly  used  defense  sites  (FUDS).  The  cost 
savings  are  expected  to  be  particularly  significant  at  removal  action  (RA)  sites.  Parsons  is 
currently  involved  with  U.S.  Army  Corps  of  Engineers  (US  ACE)  on  several  MEC  and  recovered 
chemical  warfare  materiel  (RCWM)  sites  that  could  be  used  for  additional  testing  and  refining  of 
the  process  required  for  this  type  of  discrimination  approach. 
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This  demonstration  consisted  of  the  cued  interrogation  of  previously  identified  EM61-MK2 
anomalies  using  advanced  EMI  systems.  The  advanced  EMI  MetalMapper  system  was 
demonstrated  over  6  acres  of  open  area.  Three  developmental  EMI  systems — the  Naval 
Research  Laboratory’s  2x2  TEMTADS  Array,  Lawrence  Berkeley  National  Laboratory’s 
(LBNL)  handheld  Berkeley  UXO  Discriminator  (BUD),  and  Sky  Research’s  Man-Portable 
Vector  (MPV) — were  tested  by  other  vendors  within  3  acres  of  wooded  area  and  1  acre  of  open 
area  (combined  area).  The  developmental  EMI  systems  are  discussed  in  separate  reports 
submitted  by  the  respective  demonstrators. 

1.3  REGULATORY  DRIVERS 

As  part  of  the  cleanup  of  former  DoD  sites,  buy-in  is  required  from  regulatory  agencies  at  the 
federal,  state,  and  local  levels.  The  advancement  in  classification  sensors  and  their  successful 
deployment  at  real-world  sites  needs  to  be  documented  for  their  use  to  be  accepted  by  the 
applicable  regulatory  agencies.  Their  acceptance  of  the  use  of  this  technology  at  sites  for  which 
they  are  ultimately  responsible  will  be  particularly  important  with  the  potential  for  Department 
of  Defense  (DoD)  budget  cuts  to  affect  the  amount  of  money  that  will  be  available  for  future 
remedial  actions. 


2.0  TECHNOLOGY 

2.1  TECHNOLOGY  DESCRIPTION 

The  MetalMapper  is  an  advanced  EMI  system  developed  by  Geometries,  Inc.,  with  support  from 
the  ESTCP.  The  MetalMapper  draws  elements  of  its  design  from  advanced  systems  currently 
being  developed  by  G&G  Sciences,  Inc.  (supported  by  Naval  Sea  Systems  Command,  the 
Strategic  Environmental  Research  and  Development  Program  [SERDP],  and  ESTCP)  and  by 
LBNL  with  support  from  SERDP  and  ESTCP.  It  has  three  mutually  orthogonal  transmit  loops  in 
the  Z,  Y,  and  X  directions  and  contains  seven  triaxial  receiver  antennas  inside  the  Z  (bottom) 
loop.  Typically,  the  transmit  loops  are  driven  with  a  classical  bipolar  pulse-type  time  domain 
electromagnetic  waveform  (i.e.,  alternating  pulse  polarity  with  a  50%  duty-cycle).  Depending  on 
the  survey  mode  (e.g.,  Static/Dynamic),  the  fundamental  frequency  of  transmission  can  be  varied 
over  the  range  1.1  l<f  <810  hertz  (Hz).  The  seven  receiver  antennas  allow  21  independent 
measurements  of  the  transient  secondary  magnetic  field. 

The  data  acquisition  computer  (DAQ)  is  built  around  a  commercially  available  product  from 
National  Instruments.  The  National  Instruments  DAQ  is  a  full-featured  PC  running  Windows  7. 
The  DAQ,  electromagnetic  transmitter,  and  batteries  for  the  system  are  packaged  in  an  aluminum 
case  that  can  be  mounted  on  a  pack  frame,  on  a  separate  cart  such  as  a  hand  truck,  or  on  the 
survey  vehicle  such  a  tractor.  The  instrumentation  package  also  includes  two  external  modules 
that  provide  real-time  kinematic  (RTK)  global  positioning  system  (GPS)  location  and  platform 
attitude  (i.e.,  magnetic  heading,  pitch,  and  roll)  data.  These  modules  are  connected  to  the  DAQ 
through  serial  RS232C  ports.  A  block  diagram  of  the  DAQ  system  is  in  Figure  2-1. 

The  MetalMapper  has  two  modes  of  data  collection:  dynamic  and  static.  Data  collected  in 
dynamic  mode  results  in  data  files  containing  many  data  samples.  Generally  speaking,  dynamic 
mode  data  are  collected  while  the  antenna  platform  is  in  motion.  Static  mode  data  collection  is 
employed  for  cued  surveys.  As  its  name  implies,  the  antenna  platform  remains  static  or 
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motionless  during  the  period  of  data  acquisition.  Depending  on  the  acquisition  parameters  (e.g., 
sample  period  and  stacking  parameter)  it  can  take  tens  of  seconds  to  complete  a  static 
measurement.  The  results  of  the  static  measurement  are  written  into  a  binary  data  file  containing 
only  a  single  data  point  representing  the  average  (stacked)  result,  usually  over  tens  or  even 
hundreds  of  repetitions  of  the  transmitter’s  base  frequency. 


Figure  2-1:  DAQ  and  DAQ  Functional  Block  Diagram 


A)  MM  DAQ  assembly  with  Panasonic  terminal. 


B)  Function  block  diagram  of  MM  system 


Data  are  acquired  in  time  blocks  that  consist  of  a  fixed  number  of  transmitter  cycle  “Repeats”. 
Both  the  period  (T)  and  the  repeat  factor  (N)  are  operator  selectable  and  are  varied  in 
multiplicative  factors  of  3.  The  MetalMapper  also  averages  an  operator- specified  number  of 
acquisition  blocks  (NStacks)  together  before  the  acquired  data  are  saved  to  disk.  The  decay 
transients  that  are  received  during  the  off  times  are  stacked  (averaged)  with  appropriate  sign 
changes  for  positive  and  negative  half  cycles.  The  decays  in  an  individual  acquisition  block  are 
stacked,  and  the  decays  in  that  block  are  averaged  with  other  acquisition  blocks  (assuming  the 
operator  has  selected  NStack  greater  than  one).  The  resultant  data  are  saved  as  a  data  point.  A 
photo  of  the  typical  configuration  of  the  instrument  used  for  collecting  cued  data  is  shown  in 
Figure  2-2. 


In  its  present  (third  generation)  form,  the  MetalMapper  technology  has  been  demonstrated  and 
scored  at  the  Standardized  UXO  Technology  Demonstration  sites  at  YPG  (blind  grid  only),  at 
APG  (blind  grid  plus  direct  fire  and  indirect  fire  areas),  and  most  recently  at  SLO  and  Camp 
Butner  in  connection  with  2009  and  2010  classification  studies  carried  out  by  ESTCP.  The 
performance  of  the  MetalMapper  at  these  sites  is  documented  in  formal  reports  issued  by  the 
Aberdeen  Test  Center  and  by  the  various  demonstrators  who  analyzed  the  data  collected  at  SLO 
and  Camp  Butner. 


2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

There  are  a  few  advanced  EMI  sensors  that  are  similar  to  the  MetalMapper  in  theory  and  design, 
with  the  most  comparable  being  the  TEMTADS  5x5  and  the  BUD.  The  TEMTADS  5x5  consists 
of  25  pairs  of  transmit/receive  coils  oriented  in  a  5x5  grid  pattern,  approximately  2  meters  (m)  to 
a  side.  The  BUD  is  composed  of  three  orthogonal  transmitters  and  eight  pairs  of  differenced 
receivers.  These  instruments  have  been  part  of  the  ongoing  ESTCP  classification 
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demonstrations,  and  similar  results  have  been  documented  for  all  three  during  previous  projects. 
The  main  advantage  of  the  MetalMapper  is  that  it  is  currently  commercially  available,  while  the 
other  two  advanced  EMI  sensors  are  generally  only  used  by  the  organizations  that  developed 
them. 


Figure  2-2:  Antenna  Array  and  Typical  Deployment  of  the  MetalMapper  System 


The  greatest  limitation  of  the  MetalMapper  is  its  size,  both  of  the  sensor  itself  and  of  the 
accompanying  computer,  screen  and  cables.  The  system  is  designed  primarily  for  use  in 
relatively  flat,  open  fields  and  cannot  currently  be  used  effectively  in  wooded  areas. 
Additionally,  its  effectiveness  in  areas  with  extremely  high  anomaly  densities  is  currently 
unknown,  although  data  were  collected  in  high-anomaly-density  areas  during  a  demonstration 
study  at  performed  Fort  Sill  in  late  2011.  Results  for  the  Fort  Sill  study  are  currently 
unavailable. 

3.0  PERFORMANCE  OBJECTIVES 

The  primary  performance  objectives  for  this  demonstration  include: 

•  Evaluating  if  classification  techniques  will  work  at  the  former  Camp  Beal  site; 

•  Evaluating  where  classification  techniques  will  work  at  former  Camp  Beale;  and 

•  Evaluating  the  cost  effectiveness  of  classification  techniques  in  the  areas  at  former  Camp 
Beale  where  classification  is  determined  to  be  effective. 

The  specific  performance  objectives  for  this  demonstration  are  summarized  in  Table  3-1. 

3.1  OBJECTIVE:  COMPLETE  COVERAGE  OF  THE  DEMONSTRATION  SITE 

The  reliability  of  the  survey  data  depends  on  the  extent  of  coverage  of  the  site.  This  objective 
concerns  the  ability  to  completely  survey  the  site  and  obtain  valid  data. 
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3.1.1  Metric 

The  metrics  for  this  objective  were  the  down-line  distance  between  adjacent  data  points  and  the 
maximum  coverage  gap  in  the  final,  mapped  EM61-MK2  data. 


Table  3-1:  Performance  Objectives  for  This  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Data  Collection  Objectives 

Complete  coverage  of 
the  demonstration  site 

Along  line  point 
spacing 

Survey  coverage 

•  Mapped  survey 
data 

98%  of  data  points  <= 
25cm  along  line 

>=  98%  coverage  at 
0.5m  line  spacing 

Repeatability  of 
instrument 
verification  strip 
measurements 

Amplitude  of 

electromagnetic 

anomaly 

Measured  target 
locations 

•  Twice-daily 
instrument 
verification  strip 
survey  data 

Amplitudes  ±25% 
Location  ±25cm 

Detection  of  all 
targets  of  interest 
(TOI) 

Percent  detected  of 
seeded  items 

•  Location  of  seeded 
items 

•  Anomaly  list 

100%  of  seeded  items 
detected 

Analysis  and  Classification  Objectives 

Maximize  correct 
classification  of  TOI 

Number  of  TOI 
retained 

•  Prioritized  anomaly 
lists 

•  Scoring  reports 
from  Institute  for 
Defense  Analyses 
(IDA) 

Approach  correctly 
classifies  all  TOI 

Maximize  correct 
classification  of  non- 
TOI 

Number  of  false 
alarms  eliminated 

•  Prioritized  anomaly 
lists 

•  Scoring  reports 
from  IDA 

Reduction  of  false 
alarms  by  >  50% 
while  retaining  all 

TOI 

Specification  of  no¬ 
dig  threshold 

Probability  of  correct 
classification  and 
number  of  false 
alarms  at 
demonstrator 
operating  point 

•  Demonstrator- 
specified  threshold 

•  Scoring  reports 
from  IDA 

Threshold  specified 
by  the  demonstrator  to 
achieve  criteria  above 
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Table  3-1:  Performance  Objectives  for  This  Demonstration  (Continued) 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Minimize  number  of 
anomalies  that  cannot 
be  analyzed 

Number  of  anomalies 
that  must  be  classified 
as  “Unable  to 

Analyze” 

•  Demonstrator  target 
parameters 

Reliable  target 
parameters  can  be 
estimated  for  >  98% 
of  anomalies  on  each 
sensor’s  detection  list. 

Correct  estimation  of 
target  parameters 

Accuracy  of  estimated 
target  parameters 

•  Demonstrator  target 
parameters 

•  Results  of  intrusive 
investigation 

X,  Y  <  15cm  (la) 

Z  <  10cm  (la) 

3.1.2  Data  Requirements 

Quality  control  (QC)  tools  in  Geosoft’s  Oasis  montaj  software  were  used  to  check  the  percentage 
of  points  more  than  25  centimeters  (cm)  from  each  other  and  the  total  area  of  data  gaps  in  each 
surveyed  grid. 

3.1.3  Success  Criteria 

This  objective  were  considered  to  have  been  met  if  at  least  98%  of  the  mapped  data  points  were 
within  25cm  down  line  of  each  other  and  if  at  least  98%  of  the  site  was  mapped  at  the  proposed 
line  spacing  of  50cm. 

3.2  OBJECTIVE:  REPEATABILITY  OF  IVS  MEASUREMENTS 

The  reliability  of  the  survey  data  also  depends  on  the  proper  functioning  of  the  survey 
equipment.  This  objective  concerns  the  twice-daily  confirmation  of  sensor  system  performance. 

3.2.1  Metric 

The  metrics  for  this  objective  were  the  amplitude  and  down-track  position  of  the  maxima 
obtained  from  each  of  the  twice-daily  surveys  of  the  instrument  verification  strip  (IVS). 

3.2.2  Data  Requirements 

Daily  IVS  survey  data  were  used  to  judge  this  objective. 

3.2.3  Success  Criteria 

This  objective  is  considered  met  if  the  measured  amplitude  for  each  object  in  the  IVS  was  within 
25%  of  the  expected  response  and  if  the  position  of  the  anomaly  was  within  25cm  of  the  known 
location. 

3.3  OBJECTIVE:  DETECTION  OF  ALL  TARGETS  OF  INTEREST 

Quality  data  should  lead  to  a  high  probability  of  detecting  the  TOI  at  the  site. 
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3.3.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  seed  items  detected  using  the  specified 
anomaly  selection  threshold. 

3.3.2  Data  Requirements 

The  target  list  developed  using  the  EM61-MK2  data  was  compared  to  the  known  locations  of  the 
items  seeded  during  the  project  to  judge  this  objective. 

3.3.3  Success  Criteria 

The  objective  is  considered  met  if  100%  of  the  seeded  items  were  detected. 

3.4  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 

One  of  the  two  main  objectives  of  this  demonstration  was  to  correctly  classify  all  seeded  items 
and  any  MEC  items  remaining  at  the  site  as  TOI. 

3.4.1  Metric 

The  metric  for  this  objective  was  the  number  of  items  on  the  MetalMapper  anomaly  list  correctly 
classified  as  TOI. 

3.4.2  Data  Requirements 

MetalMapper  data  were  analyzed  to  create  a  prioritized  dig  list,  which  assigned  each  target  to 
one  of  three  categories:  1)  TOI  2)  Non-TOI,  or  3)  Can’t  Analyze.  The  targets  classified  as  either 
TOI  or  Can’t  Analyze  were  considered  “dig”  targets.  The  Institute  for  Defense  Analysis  (IDA) 
used  scoring  algorithms  to  compare  the  “dig”  targets  to  the  list  of  items  identified  as  TOI  during 
the  intrusive  survey. 

3.4.3  Success  Criteria 

The  objective  is  considered  met  if  all  of  the  items  of  interest  are  correctly  labeled  as  TOI  on  the 
prioritized  anomaly  list. 

3.5  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  In  addition  to  correctly  classifying  TOI,  the  effectiveness  of  the  MetalMapper  in 
discriminating  munitions  is  a  function  of  the  degree  to  which  responses  that  do  not  correspond  to 
TOI  can  be  eliminated  from  consideration  during  the  intrusive  investigation. 

3.5.1  Metric 

The  metric  for  this  objective  was  the  number  of  targets  on  the  ranked  anomaly  list  created  using 
the  MetalMapper  data  that  were  correctly  classified  as  non-TOI. 

3.5.2  Data  Requirements 

MetalMapper  data  were  analyzed  to  create  a  prioritized  dig  list,  which  assigned  each  target  to 
one  of  three  categories:  1)  TOI  2)  Non-TOI,  or  3)  Can’t  Analyze.  The  targets  classified  as  Non- 
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TOI  were  considered  “no  dig”  targets.  IDA  used  scoring  algorithms  to  compare  the  “no  dig” 
targets  to  the  list  of  items  identified  as  Non-TOI  during  the  intrusive  survey. 

3.5.3  Success  Criteria 

The  objective  is  considered  met  if  more  than  50%  of  the  non-TOI  items  can  be  correctly  labeled 
as  Non-TOI. 

3.6  OBJECTIVE:  SPECIFICATION  OF  NO-DIG  THRESHOLD 

In  a  retrospective  analysis  as  performed  in  this  demonstration,  it  is  possible  to  tell  the  true 
classification  capabilities  of  a  classification  procedure  based  solely  on  the  ranked  anomaly  list 
submitted.  In  a  real-world  scenario,  not  all  targets  may  be  dug,  so  the  success  of  the  approach 
will  depend  on  the  ability  of  an  analyst  to  accurately  specify  their  dig/no-dig  threshold. 

3.6.1  Metric 

The  probability  of  correct  classification  (Pciass)  and  number  of  false  alarms  (Nfa)  at  the  dig/no  dig 
threshold  in  the  prioritized  dig  list  are  the  metrics  for  this  objective. 

3.6.2  Data  Requirements 

Following  data  collection,  MetalMapper  data  was  analyzed  to  create  a  prioritized  dig  list,  which 
assigned  each  target  to  one  of  three  categories:  1)  TOI,  2)  Non-TOI,  or  3)  Can’t  Analyze.  The 
category  into  which  each  target  was  placed  was  determined  using  a  decision  statistic  developed 
during  analysis  of  the  MetalMapper  data.  The  dig/no  dig  threshold  for  this  project  was  the 
decision  statistic  value  that  separated  targets  classified  as  TOI  from  those  classified  as  Non-TOI. 
IDA  personnel  used  their  scoring  algorithms  to  assess  the  results. 

3.6.3  Success  Criteria 

The  objective  is  considered  met  if  more  than  50%  of  the  non-TOI  items  can  be  correctly  labeled 
as  Non-TOI  while  retaining  all  of  the  TOI  at  the  specified  threshold. 

3.7  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  using  the  collected  MetalMapper 
data  cannot  be  classified.  These  anomalies  must  be  placed  in  the  dig  category,  which  reduces  the 
effectiveness  of  the  classification  process. 

3.7.1  Metric 

The  number  of  anomalies  for  which  reliable  parameters  cannot  be  estimated  is  the  metric  for  this 
objective. 

3.7.2  Data  Requirements 

Those  targets  for  which  parameters  could  not  be  reliably  estimated  were  identified  as  such  on  the 
prioritized  dig  list  submitted  following  analysis  of  the  MetalMapper  data. 
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3.7.3  Success  Criteria 


The  objective  is  considered  met  if  reliable  parameters  can  be  estimated  for  >  98%  of  the  targets 
on  the  prioritized  dig  list. 

3.8  OBJECTIVE:  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

This  objective  involves  the  accuracy  of  the  target  parameters  that  are  estimated  in  the  first  phase 
of  the  analysis.  Successful  classification  is  only  possible  if  the  input  features  are  internally 
consistent.  The  obvious  way  to  satisfy  this  condition  is  to  estimate  the  various  target  parameters 
accurately. 

3.8.1  Metric 

Accuracy  of  estimation  of  target  parameters  is  the  metric  for  this  objective. 

3.8.2  Data  Requirements 

The  inverted  or  fit  locations  determined  for  each  target  during  MetalMapper  analysis  and  the 
locations  of  recovered  items,  as  recorded  by  the  intrusive  teams,  were  compared  to  determine  the 
difference  between  the  two. 

3.8.3  Success  Criteria 

The  objective  is  considered  to  be  met  if  the  estimated  X,  Y  locations  are  within  15cm  (la)  of  the 
actual  locations  and  if  the  estimated  depths  are  within  10cm  (la). 

3.9  OBJECTIVE:  INTRUSIVE 

The  reliability  of  the  demonstration  depends  on  the  correct  identification  of  the  anomalies 
intrusively  investigated.  This  objective  concerns  the  ability  of  the  intrusive  personnel  to  identify 
and  document  the  anomalies  in  accordance  with  the  intrusive  plan. 

3.9.1  Metric 

The  metrics  for  this  objective  is  the  accuracy  and  precision  of  the  locations  once  they  are  located 
during  the  demonstration. 

3.9.2  Data  Requirements 

The  GPS  survey  data  collected  during  the  seed  emplacement  vs.  the  final  demonstration  analysis 
were  used  to  confirm  this  objective. 

3.9.3  Success  Criteria 

This  objective  will  be  considered  to  be  met  if  all  seeds  are  placed  in  the  ground  according  to  the 
seeding  plan  (Appendix  C)  and  a  comparison  to  the  location  recorded  by  the  intrusive  team. 


4.0  SITE  DESCRIPTION 

The  former  Camp  Beale  is  approximately  10  miles  east  of  Marysville,  California,  in  Yuba 
County.  The  demonstration  area  chosen  by  ESTCP  includes  historic  ranges  used  from  1948  to 
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1959.  Currently,  the  demonstration  area  lies  approximately  2  miles  east  of  the  active  Beale  Air 
Force  Base  installation  boundary  just  off  Waldo  Road.  The  demonstration  area  is  currently 
owned  by  the  State  of  California  and  is  managed  as  the  Spenceville  Wildlife  and  Recreation 
Area.  A  large  percentage  of  the  land  within  the  former  Camp  Beale  is  undeveloped  and  used  for 
agriculture  (primarily  livestock). 

The  demonstration  area  chosen  by  ESTCP  includes  historic  ranges  used  from  1948  to  1959. 
Based  on  archives  search  report  findings,  ranges  on  Camp  Beale  included  the  use  of  light, 
medium,  and  heavy  tanks;  self-propelled  anti-tank  gun;  37mm  anti-aircraft  guns;  105mm 
howitzers;  81mm  mortars;  and  4.2  inch  mortars.  Ranges  and  bombing  targets  surrounding  the 
demonstration  area  include  small  arms  (Ranges  6,  11,  and  12)  and  ground  to  air  artillery  ranges 
(Range  13)  and  three  overlapping  bombing  targets  that  were  used  by  both  the  Air  Force  and 
Navy. 

In  1959,  the  U.S.  Government  declared  the  portions  of  Beale  Air  Force  Base  as  excess,  and  a 
surface  clearance  was  conducted  prior  to  the  transfer  of  the  property.  The  property  was 
transferred  to  the  State  of  California  between  1962  and  1969  and  was  recommended  for  surface 
use  only.  Currently,  the  area  consists  of  the  Spenceville  Wildlife  and  Recreation  Area. 

An  aerial  photo  of  former  Camp  Beale  and  the  demonstration  area  is  shown  in  Figure  4-1. 

4.1  SITE  SELECTION 

The  former  Camp  Beale  was  chosen  based  on  partially  wooded  terrain,  steep  slopes,  and  a  wide 
mixture  of  munitions,  which  increased  the  complexity  of  the  classification  process.  The  first  site 
in  the  series,  former  Camp  Sibert  in  Alabama,  had  only  one  TOI,  and  item  “size”  was  an 
effective  discriminant.  A  hillside  range  at  the  former  SLO  in  California  was  selected  for  the 
second  of  these  demonstrations  because  of  the  wider  mix  of  munitions,  including  60mm,  81mm, 
and  4.2-in.  mortars  and  2.36-in.  rockets.  The  third  site  chosen  was  the  former  Camp  Butner  in 
North  Carolina,  which  was  known  to  be  contaminated  with  items  as  small  as  37-mm  projectiles, 
adding  yet  another  layer  of  complexity  into  the  process. 

4.2  GEOLOGY 

The  principal  physiographic  units  in  the  area  are  the  dissected  alluvial  uplands  west  of  the  Sierra 
Nevada,  the  foothills  section  of  the  Sierra  Nevada,  and  the  Sierra  Nevada  Mountains.  The 
dissected  alluvial  uplands  consist  of  low  hills  and  gently  rolling  country  that  merge  with  the 
foothills  of  the  Sierra  Nevada  on  the  east  and  the  low  alluvial  plains  of  eastern  Sacramento 
Valley  on  the  west.  The  foothills  lie  to  the  east  of  the  alluvial  uplands  and  are  an  undulating  to 
very  steep  region  at  the  base  of  the  Sierra  Nevada  Mountains.  Geologic  units  outcropping  in  the 
area  include  the  basement  complex  of  metamorphosed  igneous  and  sedimentary  rocks  and 
igneous  batholiths  and  intrusives,  andesitic  and  rhyolitic  volcanic  rocks,  the  Laguna  Formation 
and  related  continental  deposits  of  Pliocene  and  Pleistocene  age,  the  Victor  Formation  and 
related  continental  deposits  of  Pleistocene  age,  and  river  deposits  of  Holocene  age  (USACE, 
1997). 

The  three  primary  near-surface  soil  profiles  are  related  to  geomorphic  location:  alluvial  upland, 
Sierran  foothills,  and  mountain  series.  The  alluvial  uplands  in  the  area  are  underlain  by  silty 
sands  with  gravel,  which  cover  the  weathered  granite  bedrock  surface.  The  surface  soil  consists 
of  approximately  15  inches  of  grayish- brown,  coarse-grained  silty  sand  with  gravel.  The  subsoil 
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is  pale-brown,  coarse-grained  silty  sand  with  gravel.  The  highly  weathered  granodiorite  is  about 
30  inches  below  the  surface.  Permeability  is  moderately  rapid.  The  foothills  of  the  Sierra  Nevada 
are  underlain  by  sandy  and  gravelly  silts,  covering  vertically  tilted  metamorphic  rock.  Typically, 
the  surface  layer  of  the  soil  is  a  brown  gravelly  silt  with  sand,  about  4  inches  thick.  The  subsoil  is 
yellowish  red  gravelly  silt  with  sand.  Metamorphic  schist  bedrock  typically  is  found  at  a  depth  of 
about  20  inches  (US ACE,  1997). 

4.3  MUNITIONS  CONTAMINATION 

The  suspected  munitions  in  this  demonstration  area  include  but  are  not  limited  to: 

•  37mm  projectiles 

•  60mm  mortars 

•  81mm  mortars 

•  105  mm  projectiles 

At  the  site  of  this  demonstration,  evidence  of  81mm  mortars  and  105mm  projectiles  was  found 
during  the  site  inspection  intrusive  investigation  in  2005.  It  is  also  suspected  that  60mm  mortars 
may  be  present.  In  addition,  37mm  projectiles  have  been  found  scattered  throughout  the  former 
Camp  Beale  and  are  included  as  another  suspected  munition  in  this  area.  Due  to  the  complex 
historical  usage  of  this  site  over  many  years  and  the  overlapping  network  of  historical  ranges 
throughout,  it  is  also  likely  that  other  munitions  types  beyond  those  listed  above  may  be 
encountered. 

4.4  SITE  CONFIGURATION 

The  demonstration  site  covers  approximately  9  acres.  The  EM61-MK2  was  used  to  survey  the 
9-acre  site  with  100%  coverage.  The  50-acre  demonstration  area  is  shown  in  Figure  4-2. 
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5.0  TEST  DESIGN 


5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  objective  of  this  program  was  to  demonstrate  a  method  for  the  use  of  classification  in  the 
munitions  response  process.  The  three  key  components  of  this  method  were  collection  of  high- 
quality  geophysical  data  and  principled  selection  of  anomalous  regions  in  those  data;  analysis  of 
the  selected  anomalies  using  physics-based  models  to  extract  target  parameters  such  as  size, 
shape,  and  materials  properties;  and  the  use  of  those  parameters  to  construct  a  ranked  anomaly 
list.  Each  of  these  components  was  handled  separately  in  this  program. 

The  ESTCP  Program  Office  coordinated  data  collection  activities.  This  included  all  preparatory 
activities,  arranging  for  data  collection  by  well-validated  systems,  preprocessing  of  the  data  to 
include  positioning  of  each  sensor  reading  and  any  overall  filtering  required,  oversight  of 
selection  of  anomalies  for  analysis  from  each  geophysical  data  set,  and  compilation  of  a  master 
anomaly  list. 

Data  analysis  demonstrators  were  provided  with  the  survey  data  and  data  from  as  many  cued 
targets  as  they  choose.  They  processed  individual  data  sets  using  existing  routines  to  extract 
target  parameters.  These  parameters  were  passed  to  the  classification  routines  which,  after 
training  on  a  limited  amount  of  site-specific  ground  truth,  were  used  to  produce  prioritized 
anomaly  lists. 

Validation  digging  was  coordinated  by  the  Program  Office.  Since  this  was  a  demonstration,  all 
anomalies  on  the  master  anomaly  list  were  investigated.  The  underlying  target  was  uncovered, 
photographed,  located  with  a  cm-level  GPS  system,  and  removed.  Each  analysis  demonstrator 
was  able  to  request  ground  truth  data  for  training,  either  the  ground  truth  from  a  small  number  of 
grids  chosen  by  the  Program  Office  or  from  a  subset  of  the  sensor  anomaly  list  of  the 
demonstrator’s  choosing.  At  the  conclusion  of  training,  each  demonstrator  submitted  an  initial 
ranked  anomaly  list  for  each  data  set  analyzed.  These  lists  were  ordered  from  the  item  the 
demonstrator  was  most  confident  was  a  munitions  through  the  item  the  demonstrator  was  most 
confident  was  not  hazardous  and  indicated  the  threshold  the  demonstrator  chose  for  initial 
digging.  Anomalies  for  which  the  demonstrator  could  not  extract  meaningful  parameters  were 
placed  at  the  top  of  the  list.  Dig  results  from  the  first  round  of  digging  were  provided  to  each 
deomstrator,  following  which  the  demonstrators  construct  a  final  ranked  anomaly  list  with  each 
anomaly  marked  dig  or  no-dig.  These  final  inputs  were  scored  by  the  IDA,  with  emphasis  on  the 
number  of  items  that  are  correctly  labeled  nonhazardous  while  correctly  labeling  all  TOI. 

The  primary  objective  of  the  demonstration  was  to  assess  how  well  each  demonstrator  ordered 
their  ranked  anomaly  list(s)  and  specified  the  threshold  separating  high  confidence  clutter  from 
all  other  items.  The  secondary  objective  was  to  determine  the  classification  performance  that 
could  be  achieved  by  each  approach  through  a  retrospective  analysis. 

5.2  SITE  PREPARATION 

5.2.1  Acquire  Site-Specific  Information 

The  demonstration  site  falls  within  the  historical  bombing  Target  4  and  the  Proposed  Toss  Bomb 
target  area.  As  stated  above,  evidence  of  81mm  mortars  and  105mm  projectiles  was  recovered  as 
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part  of  the  site  inspection  intrusive  investigation  in  the  demonstration  area,  and  it  was  also 
suspected  that  60mm  mortars  might  have  been  present.  In  addition,  37mm  projectiles  have  been 
found  scattered  throughout  the  former  Camp  Beale,  so  they  were  included  as  another  suspected 
munition.  Due  to  the  complex  historical  usage  of  this  site  over  many  years  and  the  overlapping 
network  of  historical  ranges  throughout,  it  was  thought  possible  that  other  munitions  types 
beyond  those  listed  above  may  have  been  encountered. 

5.2.2  First-Order  Navigation  Points 

To  avoid  confusion  between  and  among  various  demonstrators,  it  was  important  that  all  survey 
data  and  validation  activities  were  conducted  on  a  common  coordinate  system.  Two  first-order 
survey  monuments  were  installed  at  the  site.  The  points  are  labeled  1  and  2,  their  coordinates  are 
given  in  Table  5.1. 


Table  5-1:  Geodetic  Control  Locations 


ID 

Northing 

Easting 

Elevation 

NAVD88 

(m) 

Beale  1 

4331339 

646936.3 

100 

Beale  2 

4331030 

647071.7 

100 

5.2.3  Surface  Clearance 

As  part  of  the  Camp  Beale  ESTCP  study,  Parsons  UXO  personnel  conducted  instrument-aided 
surface  clearance  before  the  digital  mapping  (DGM)  surveys.  The  main  objective  of  the  surface 
clearance  was  to  ensure  that  no  hazardous  items  would  be  encountered  before  the  nonintrusive 
phases  in  the  demonstration  area  and  to  remove  metallic  surface  debris  from  the  grids.  In 
addition  to  the  surface  clearance,  Parsons  also  conducted  minor  brush  clearing,  cutting  low-lying 
branches  and  removed  fallen  trees  from  the  demonstration  area.  This  operation  lasted  one  week 
starting  on  April  19,  2011.  The  majority  of  items  found  on  the  surface  sweeps  were  barbed  wire, 
along  with  small  munitions  fragments.  Two  notable  items  identified  during  the  surface  clearance 
were  an  empty  75mm  projectile  and  a  large  pile  of  barbed  wire  that  was  eventually  moved  out  of 
the  survey  area.  These  are  shown  in  Figures  5-la  and  5-lb. 

5.2.4  EM61-MK2  Coverage  Survey 

DGM  data  were  collected  using  a  hand-towed  EM61-MK2  time-domain  electromagnetic  sensor. 
Positioning  was  accomplished  using  either  RTK  GPS,  Trimble  Robotic  Total  Station  (RTS),  or 
fiducial  methods,  depending  on  the  availability  GPS  signal  and  tree  density  in  the  wooded  area. 
The  EM61-MK2  was  set  up  in  the  four-channel  mode. 
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Figure  5-1:  Items  Identified  during  Surface  Clearance 

A:  Empty  75mm  Projectile  B:  Barbed  Wire 


The  standard  EM61-MK2  consists  of  two  0.5m  by  lm  coils,  separated  vertically  by  a  distance  of 
30cm,  set  on  a  pair  of  wheels.  For  this  project,  wheels  were  used  with  the  instrument,  which  held 
the  bottom  coil  0.4m  above  the  ground.  The  EM61-MK2  sensors  were  set  to  record  four  time 
gates  from  the  bottom  coil. 

The  GPS  equipment  used  to  collect  positional  data  included  a  Trimble  R8  antenna  and  base 
station.  The  rover  collected  positional  measurements  at  a  rate  of  1  Hz  and  transmitted  them  to 
the  EM61-MK2  data  logger  or  field  computer. 

DGM  data  were  collected  using  grid-based  system  in  both  open  fields  and  wooded  areas  on 
individual  30m  by  30m  grids.  Grid-based  data  were  collected  in  parallel  lines  spaced  0.5  m 
apart  and  were  intended  to  cover  100  percent  of  the  area  being  investigated.  GPS  data  were  used 
on  the  majority  of  the  demonstration  areas  where  a  GPS  signals  were  not  affected  by  tree 
density.  Fiducial  and  Trimble  RTS  was  used  in  the  denser  wooded  areas. 

5.2.5  Seed  Operation 

At  a  live  site  such  as  this,  the  ratio  of  clutter  to  TOI  is  such  that  only  a  small  number  of  TOI  may 
be  found  in  the  investigation  area;  far  from  enough  to  determine  classification  performance  with 
acceptable  confidence  bounds.  To  avoid  this  problem,  the  site  was  seeded  with  enough  TOI  to 
ensure  reasonable  statistics. 

Parsons  conducted  seeding  operations  at  former  Camp  Beale  on  April  25,  2011,  through  May  6, 
2011.  The  seeding  locations  covered  approximately  9  acres  in  five  separate  areas  at  the  site  in 
both  open  pasture  and  wooded  grids.  The  location  of  each  seed  item  was  established  with  a 
Trimble  5800  RTK  GPS  system  and  a  Trimble  RTS  for  the  wooded  areas.  The  Base  Station 
control  point  used  for  this  operation  was  established  previously  in  Universal  Transverse 
Mercator  (UTM)  Zone  10N,  WGS84  coordinates.  All  seed  items,  with  the  exception  of  small 
industry  standard  object  (ISO),  items  were  received  from  the  Army  Research  lab  in  Maryland. 

Parsons  flagged  200  locations  with  the  Trimble  GPS  and  RTS  units  and  established  anomaly 
avoidance  at  each  location  to  ensure  a  clean  area  for  emplacement.  All  200  seed  locations  were 
dug  to  proper  size  and  depth.  Digging  operations  involved  both  mechanical  and  manual 
procedures  to  meet  exact  specifications  and  to  minimize  burial  evidence.  Prior  to  emplacement, 
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magnetic  north  was  determined.  Once  magnetic  north  was  established,  the  seed  item  was 
positioned  with  the  nose  pointing  to  the  exact  azimuth  and  dip  angle  specified  by  Parsons.  The 
dip  angle  specifications  were  set  to  a  45-degree  tolerance,  with  the  exact  angles  measured  with  a 
level.  Exact  angles  above  horizontal,  below  horizontal  were  recorded.  After  all  the  emplacement 
requirements  of  depth,  inclination,  dip  angle,  length,  and  location  were  completed,  a  photo  was 
taken  of  the  seed  item  in  the  burial  location.  All  the  emplacement  information  along  with  the 
seed  item  and  north  direction  is  visible  in  the  photos. 

Seed  location  holes  were  not  backfilled  until  the  final  QC  check  was  completed.  This  consisted 
of  comparing  the  location  with  the  original  designated  location,  capturing  the  center  location  of 
the  emplaced  seed  item  with  GPS,  and  checking  the  depth,  inclination,  and  dip  angle  of  each 
seed  item.  Once  these  checks  were  accomplished,  the  backfilling  was  started  with  a  shovel  to 
prevent  any  excess  movement  of  the  seed  items. 

Seed  items  for  the  former  Camp  Beale  Area  included  37mm  projectiles,  60mm  projectiles, 
81mm  projectiles,  105mm  projectiles  and  small  ISO  items.  Table  5-2  lists  the  seed  items. 


Table  5-2:  Seeded  Items 


Seed  item 

Total 

Small  ISO 

65 

37mm 

59 

60mm 

34 

81mm 

33 

105  mm 

9 

Total 

200 

5.2.6  Establish  an  Instrument  Verification  Strip  and  Training  Pit 

A  clean  area  based  on  EM61-MK2  data  was  located  in  grid  M05  as  the  IVS  area;  this  area  was 
flat  and  equidistant  to  the  demonstration  grid  areas.  A  50-foot  (ft.)  by  150ft.  area  was  used  to 
establish  the  IVS  and  test  pit  used  for  daily  verification  of  proper  sensor  operation  and 
calibration  to  be  used  to  collect  sensor  data  for  algorithm  training.  Table  5-3  lists  the  IVS  seed 
items. 


Table  5-3:  IVS  Items 


IVS  item 

Depth 

(cm) 

Orientation 

Shot  put 

30 

Horizontal 

105mm  HEAT  (high  explosive 
anti-tank)  missile 

45 

Horizontal 

60mm  w/o  fin 

15 

Horizontal 

37mm 

15 

Horizontal 

lin.  x  4in.  pipe  nipple 

15 

Horizontal 
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5.3  SYSTEM  SPECIFICATION 

The  MetalMapper  sensor  and  data  acquisition  system  are  described  in  detail  in  Section  2.1. 
During  the  demonstration  at  Camp  Beale,  the  antenna  array  was  placed  in  a  wooden  sled 
attached  to  the  rear  three-point  hitch  of  a  Kabota  tractor  (Figure  2-2).  A  Trimble  R8  GPS  was 
mounted  directly  above  the  sensor  array  using  a  wooden  tripod,  and  an  inertial  measurement  unit 
was  attached  to  the  wooden  support  used  to  stabilize  the  X-  and  Y-direction  transmitters,  also 
directly  above  the  center  of  the  array.  These  instruments  streamed  positional  data  constantly,  at 
a  rate  of  approximately  10Hz,  although  the  stream  rate  for  the  GPS  was  modified  throughout  the 
project  in  an  attempt  to  solve  repeated  software  crashes.  The  two  instruments  were  connected  to 
the  DAQ  via  USB  (universal  serial  bus)  ports,  and  the  incoming  data  were  used  both  to  navigate 
from  point  to  point  and  to  locate  the  collected  data. 

5.4  CALIBRATION  ACTIVITIES 
5.4.1  Test  Pit  and  IVS  Data  Collection 

A  test  pit  was  constructed  at  the  site  before  the  arrival  of  the  MetalMapper  data  collection  team. 
The  pit  was  an  approximately  3ft.  by  3ft.  by  3ft.  hole  that  allowed  the  collection  of  static 
MetalMapper  data  over  TOI  items  expected  at  the  site.  The  test  pit  data  could  then  be  used  for 
comparison  with  field  data  collected  over  unknown  targets.  Test  pit  items  were  generally 
oriented  in  four  directions  relative  to  the  MetalMapper  at  two  different  depths  for  a  total  of  eight 
readings  per  item.  Table  5-4  indicates  the  various  measurements  collected  in  the  test  pit. 


Table  5-4:  Data  Collected  in  the  Test  Pit 


Item  ID 

Depths 

(cm) 

Orientations 

Empty  pit  (static  -  5  min) 

N/A 

N/A 

Steel  sphere 

27 

N/A 

37mm  projectile 

10,  20 

Horizontal  (along  track  and  across  track),  45° 
down,  90°  down 

60mm  mortar 

20,  30 

Horizontal  (along  track  and  across  track),  45° 
down,  90°  down 

81mm  mortar 

25,45 

Horizontal  (along  track  and  across  track),  45° 
down,  90°  down 

105mm  projectile 

30,  45 

Horizontal  (along  track  and  across  track),  45° 
down,  90°  down 

Small  industry  standard  object 
(ISO) 

20 

Horizontal  (along  track  and  across  track),  45° 
down,  90°  down 

Flare  casing  (native) 

25 

Horizontal  (along  track  only),  45°  down,  90°  down 

75mm  (native;  blown  open) 

25 

Horizontal  (along  track  only),  45°  down,  90°  down 

Unknown  fragment  (native) 

25 

Horizontal  (no  specific  orientation ),  90°  down 

In  addition  to  the  test  pit  data,  data  were  collected  over  one  of  two  IVSs  twice  daily.  One  of  the 
IVSs  was  in  the  central  portion  of  the  site  and  was  constructed  before  the  MetalMapper  team’s 
arrival.  This  strip  consisted  of  five  items  of  various  sizes,  a  steel  sphere,  small  ISO,  37mm 
projectile,  60mm  mortar,  and  105mm  projectile.  It  was  intended  that  this  strip  be  used 
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throughout  the  project,  but  the  drive  time  between  this  IVS  and  most  of  the  MetalMapper  targets 
on  the  west  side  of  an  intervening  hill  was  deemed  excessive.  To  save  time  driving  between  the 
IVS  and  the  targets  and  to  avoid  undue  stress  on  the  MetalMapper  during  the  drive,  Parsons  set 
up  another  IVS  on  the  west  side  of  the  site  near  the  survey  area.  This  second  IVS  consisted  of 
three  items:  a  37mm  projectile,  60mm  mortar,  and  81mm  mortar.  IVS  data  were  collected  at  one 
of  the  two  strips  before  and  after  daily  data  collection.  All  data  collected  over  the  IVS  strip  were 
inverted  in  the  field  as  described  in  Section  6.2  and  compared  to  the  Camp  Beale  target  library  as 
described  in  Section  6.4.  Two  tests  were  performed  using  the  IVS  data: 

•  Inverted  locations  were  compared  to  the  known  locations  for  the  IVS  seed  items,  with  the 
differences  between  the  modeled  and  known  locations  expected  to  be  less  than  15cm  X,Y 
and  10cm  depth;  and 

•  The  item  identified  by  the  target  library  comparison  was  compared  to  the  actual  buried 
item,  and  it  was  expected  that  the  identified  item  matched  TOI  with  a  confidence  high 
enough  that  it  would  be  marked  as  a  dig  (0.7  confidence  expected  in  the  field) 

All  IVS  data  met  the  above  requirements. 

5.4.2  Background  Data 

Background  data  were  collected  at  least  twice  daily,  typically  before  each  survey  of  the  IVS 
strip.  Daily  background  data  were  collected  at  designated  locations  adjacent  to  the  IVS  strip. 
Additional  background  points  were  collected  at  the  operators’  discretion  during  the  day  if  they 
felt  another  point  was  necessary  for  any  reason  (changes  to  the  configuration  of  the  DAQ  to 
minimize  software  crashes,  changing  field  conditions  such  as  rain,  and  etc.).  One  of  the  lessons 
learned  from  this  project  was  that  background  measurements  should  be  collected  more  often  to 
account  for  potential  changes  such  as  drying  dew  throughout  the  morning  and  across-site 
differences  in  soil  and/or  geology.  Parsons  introduced  more  frequent  background  collection  for 
later  projects. 

5.5  DATA  COLLECTION  PROCEDURES 

The  operator  moved  the  array  by  lifting  the  sled,  navigating  to  the  vicinity  of  each  selected  point 
using  the  graphic  display  on  the  computer  monitor  in  front  of  him,  and  setting  the  MetalMapper 
down  on  the  point.  Reacquisition  of  the  EM61  targets  selected  for  cued  data  collection  was 
accomplished  using  “dancing  arrows”  displayed  on  the  monitor.  The  “dancing  arrows”  display 
shows  the  seven  receivers  in  the  array,  arranged  as  they  are  in  the  Z-coil,  typically  with  a  blue 
arrow  pointing  out  of  each.  The  arrows  point  toward  the  metallic  source  nearest  each  of  the 
receivers.  Under  ideal  conditions,  there  is  one  source  in  the  vicinity  of  the  selected  point,  and  all 
of  the  arrows  point  inward  toward  the  center  of  the  array.  In  the  case  of  multiple  sources,  one  or 
more  of  the  outer  arrows  may  point  outward  from  the  array  toward  another  piece  of  metal. 
Generally,  the  operator  attempted  to  position  the  array  such  that,  at  least,  the  arrows  in  the  three 
receivers  closest  the  middle  of  the  coil  were  pointing  at  each  other. 

The  MetalMapper’ s  single-point  or  cued-collection  mode  was  used  for  all  data  collection  at 
Camp  Beale.  Once  the  MetalMapper  was  positioned  correctly  above  the  target,  the  operator 
collected  a  data  point  using  the  settings  indicated  in  Table  5-5. 
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Table  5-5:  Acquisition  Parameters  Used  during  the  Camp  Beale  Demonstration 


Mode 

Tx 

Mode 

Hold-Off 
Time  (us) 

El S 

Rep  Fctr 

Dec 

Fctr 

(%) 

Stk 

Const 

Decay 
Time  (us) 

No. 

Gates 

Sample 
Period  (s) 

Sample 
Rate  (S/s) 

Static 

ZYX 

50 

0.9 

27 

10 

10 

30 

8333 

50 

9 

N/A 

Static  targets  were  identified  according  to  the  ID  determined  for  each  target  picked  in  the 
dynamic  EM61-MK2  survey.  In  the  case  of  repeated  measurements  associated  with  a  single 
target  point,  10,000  was  added  to  the  original  ID  (e.g.,  the  re-shot  for  0001  was  10001). 


5.5.1  Scale  of  Demonstration 

Parsons’  field  team  collected  1,729  data  points  during  the  project  for  1,491  targets.  Additional 
points  were  either  re-shots  of  already  collected  points  due  to  a  high  offset  between  the  location  of 
the  center  of  the  MetalMapper  and  the  modeled  location  of  a  target  or  for  points  that  may  have 
been  incorrectly  identified  during  collection.  A  distance  of  40cm  between  the  array  location  and 
the  modeled  location  was  considered  the  greatest  acceptable  distance  between  the  two  points. 
Re-shots  were  collected  for  any  targets  with  a  larger  offset. 

5.5.2  Sample  Density 

One  data  point  was  collected  per  target,  as  described  in  Section  5.5;  re-shots  were  collected  for 
any  targets  that  did  not  invert  correctly  during  processing  or  for  targets  with  modeled  locations 
greater  than  40cm  from  the  collection  location. 

5.5.3  Data  Quality  Checks 

An  instrument  calibration  check  was  conducted  a  minimum  of  twice  a  day  (at  the  beginning  and 
the  end  of  the  field  day),  and  the  quality  of  static  data  points  were  reviewed  visually  using  the 
plotting  capabilities  of  the  acquisition  software.  These  checks  ensured  that  the  instrumentation 
was  functional,  properly  calibrated,  and  stable. 

A  final  check  on  the  quality  of  static  data  was  performed  after  initial  inversion  was  performed 
using  the  UX-Analyze  module  in  Oasis  montaj.  Inverted  target  locations  were  compared  to  data 
collection  locations  to  determine  if  offsets  between  the  two  are  greater  than  40cm.  Re-shots 
were  collected  for  all  targets  with  offsets  greater  than  40cm. 

5.5.4  Data  Handling 

Data  were  recorded  in  binary  format  as  files  on  the  hard  disk  of  the  MetalMapper  DAQ.  These 
data  were  offloaded  to  other  media  at  least  once,  and  sometimes  more  frequently,  per  day.  The 
computer’s  hard  disk  had  enough  capacity  to  store  all  the  data  from  the  entire  site,  so  these  data 
were  not  erased  until  they  had  been  thoroughly  reviewed  and  archived.  The  data  file  names 
acquired  each  day  were  cataloged  (usually  on  a  spreadsheet)  and  integrated  with  any  notes  or 
comments  in  the  operator’s  field  book.  All  data  ended  up  on  the  hard  drives  of  one  or  more 
laptop  computers  used  to  post-process  data.  Data  were  also  archived  to  a  data  server  in  the 
Parsons  office. 

Raw  binary  files  were  preprocessed  using  the  TEM2CSV  software  package,  which  outputs 
“preprocessed  and  located”  data  files  in  a  text  readable  format  (.CSV).  Preprocessing  included 
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the  location  of  the  point  in  UTM  meters  and  subtraction  of  background.  Located  and 
background  corrected  .CSV  files  were  imported  into  Oasis  montaj  for  further  processing  and 
analysis. 

5.6  INTRUSIVE  PROCEDURES 

Parsons’  intrusive  operations  at  former  Camp  Beale  began  in  July  11,  2011,  and  ended  on 
August  22,  2011.  Operations  began  with  the  site-specific  training,  which  included  prepping  the 
staging  area  for  intrusive  activities  and  performing  equipment  checks.  The  staging  area 
consisted  of  a  20ft. -long  metallic  storage  unit  and  a  mobile  office  at  the  Spenceville  Wildlife 
compound.  All  Parsons  intrusive  equipment  was  stored  at  the  compound  and  locked  at  the  end 
of  the  day.  Daily  equipment  check  included  confirming  GPS  accuracy  over  known  monuments, 
EM61-Mk2  static  tests,  and  handheld  analog  instruments  calibrations. 

Intrusive  investigation  of  the  2,143  anomalies  started  in  grid  R09  and  concluded  on  grid  B02. 
All  2,143  anomalies  were  intrusively  investigated  and  documented  per  the  demonstration  work 
plan.  All  excavated  anomalies,  excluding  the  seed  items,  were  placed  in  a  sandbag  and  later  in  a 
plastic  55-gallon  drum  for  storage.  Seven  55-gallon  drums  were  filled  with  the  sandbags, 
weighing  842  pounds.  Seed  items  intrusively  investigated  were  stored  in  a  separate  bin  and 
inventoried  daily.  Once  all  the  seed  items  were  accounted  for,  they  were  shipped  off  site. 

Personnel  on  site  to  conduct  the  intrusive  operation  included  Parsons  and  Ordnance  & 
Explosives  Remediation  (OER),  the  UXO  explosives  subcontractor.  The  field  team  consisted  of 
six  Parsons  Personnel  and  two  OER  personnel.  Parsons’  site  safety  and  health  manager  and  the 
Parsons’  site  manager  conducted  daily  site  safety  briefings,  as  appropriate. 

5.6.1  Equipment 

The  equipment  used  during  the  Camp  Beale  intrusive  activities  included  the  following: 

•  Schonstedt  Magnetic  Locator  (Model  GA-52Cx) 

•  Whites  Metal  detector 

•  EM61-MK2 

•  Trimble  5800  GPS  system 

•  Digital  level 

•  Miscellaneous  hand  tools 

•  Digital  cameras 

5.6.2  Field  Procedures 

Reacquisition  of  all  targets  was  conducted  using  the  Trimble  5800  GPS  system.  The  Trimble 
5800  GPS  base  station  was  set  up  on  survey  monument  Beale  1  and  checked  daily  on  monument 
Beale  2.  Parsons  flagged  all  target  locations  with  a  plastic  pin  flag  that  was  marked  with  the 
target  identification  and  EM61-MK2  pre- value.  The  depth  estimations  determined  the  initial 
approach  to  every  target.  To  preserve  the  azimuth  and  inclination  of  the  anomalies,  the  digging 
of  all  targets  began  with  skinning  the  surface  manually  by  hand.  Azimuth  data  reflected  the 
magnetic  north.  Inclination  was  determined  by  a  digital  level  and  readings  reflected  positive  or 
negative  from  horizontal.  Location  data  captured  by  GPS  was  used  to  document  the  center  mass 
and  elevation  of  each  item.  A  photograph  was  collected  of  the  item  with  written  dig  result  data 
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on  a  whiteboard.  Lastly,  an  EM61-MK2  unit  was  used  to  scan  the  location  to  confirm  the 
absence  of  all  metallic  items  from  that  target  location. 

The  Parsons  team  leader  who  orchestrated  the  movements  of  the  different  tasks  associated  with 
the  information-gathering  process  recorded  all  documentation  on  Trimble  Geo  XT  units.  The 
intrusive  operations  consisted  of  two  intrusive  teams.  Each  team  was  responsible  for 
reacquisition,  intrusive,  and  anomaly  documentation.  Once  enough  anomalies  were  processed, 
the  least-busy  team  conducted  the  EM61-MK2  QC  over  the  excavated  holes. 

Munitions  debris  (MD)  and  cultural  debris  (CD)  scrap  collected  from  target  locations  was  stored 
in  labeled  burlap  sandbags  with  the  pin  flag.  Parsons’  senior  UXO  and  site  safety  officer 
certified  all  MD  scrap  by  thoroughly  going  through  each  piece  individually  before  final 
disposition  in  the  sealed  bins. 

All  seed  items  recovered  from  intrusive  operations  were  stored  in  a  secure  area  and  prepared  for 
final  shipment.  The  seed  items  were  shipped  on  October  4,  2011,  to  the  Army  Research  Lab  in 
Welcome,  Maryland.  The  shipments  included  certified  and  signed  DoD  1348  forms  and 
additional  letters  signed  by  Parsons’  senior  UXO  personnel. 

All  target  locations  were  backfilled  after  completion  of  the  excavation.  Once  the  final 
anomalies  were  excavated  and  backfilled,  Parsons  conducted  a  walkthrough  and  confirmed  that 
all  holes  were  filled  and  no  trash  was  left. 

Excavation  data  collected  by  the  intrusive  team  was  digitally  downloaded  to  a  database  and 
reviewed  daily.  The  daily  information  required  the  target  ID  to  be  connected  with  intrusive 
documentation,  photo,  and  GPS  coordinates.  Assessment  of  each  target  item  required  the 
coordinates  to  match  the  original  location  and  the  picture  to  match  the  documented  findings. 
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Figure  5-3:  Intrusive  Operation  Photos 
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5.7  VALIDATION 


As  part  of  the  intrusive  operation,  Parsons  investigated  2,143  anomalies,  which  included  200 
seed  items.  The  average  completed  digs  per  day  were  93,  based  on  23  intrusive  days.  The 
majority  of  the  MD  encountered  consisted  of  unidentified  munitions  fragments.  No  MEC  items 
were  discovered  during  the  intrusive  operation.  The  anomaly  breakdown  in  the  study  area  was 
as  follows: 


Table  5-6:  Intrusive  Results 


Type 

Anomalies 

Overall  Results(9  acres) 

Cultural  Debris 

194 

Munitions  Debris 

1710 

No  Contact 

39 

Seed 

200 

Open  Area  Grid  (5  acres) 

Cultural  Debris 

173 

Munitions  Debris 

944 

No  Contact 

3 

Seed 

112 

Combined  Grids  (1  acre) 

Cultural  Debris 

3 

Munitions  Debris 

236 

No  Contact 

0 

Seed 

20 

Wooded  Grids  (3  acres) 

Cultural  Debris 

18 

Munitions  Debris 

530 

No  Contact 

36 

Seed 

68 

5.8  MUNITIONS  DEBRIS  SCRAP 

MD  and  CD  scrap  recovered  from  the  demonstration  area  at  the  former  Camp  Beale  amounted  to 
approximately  842  pounds.  The  MD/CD  scrap  filled  six  55-gallon  drums,  all  with  locks  and 
numbered  custody  seals.  Shipping  information,  along  with  certified  and  signed  DOD  1348 
forms,  was  included  with  the  final  transfer  of  the  drums.  YRC  shipping  picked  up  the  drums  on 
October  6,  2011.  The  shipment  included  the  eight  55-gallon  drums  and  six  wooden  crates,  which 
included  the  inert  105mm  projectiles.  The  shipment  was  delivered  to  Glen  Harbaugh  at  the 
Army  Research  Lab  in  Welcome,  Maryland.  The  total  weight  of  the  shipment  was  1,690 
pounds. 
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6.0  DATA  ANALYSIS  AND  PRODUCTS 

The  MetalMapper  was  used  to  collect  static  data  over  1,491  targets  (including  30  test  targets  not 
intended  for  inclusion  on  the  ranked  dig  list)  identified  at  former  Camp  Beale  based  on  EM61- 
MK2  data.  The  processing  and  analysis  steps  that  were  used  to  generate  a  dig/no  dig  decision  for 
each  target  are  described  below. 

6.1  PREPROCESSING 

Raw  MetalMapper  data  are  collected  and  stored  as  .tern  files.  The  MetalMapper  acquisition 
software  uses  a  convention  for  assigning  a  unique  name  to  each  data  file  without  the  need  to 
manually  enter  the  name.  The  operator  supplies  a  prefix  for  the  root  name  of  the  file  (e.g., 
“Static”).  The  acquisition  software  then  automatically  appends  a  5-character  numerical  index  to 
the  filename  prefix  to  form  a  unique  root  name  for  the  data  file  (e.g.,  StaticOOOOl).  The  index  is 
automatically  incremented  after  the  file  has  been  successfully  written.  Although  the  Target 
identification  (ID)  is  not  used  as  the  file  name  in  the  .tern  file,  the  Target  ID  is  stored  in  the  file 
according  to  name  of  the  target  highlighted  on  the  MetalMapper  screen  during  collection. 

Preprocessing  of  the  .tern  files  was  accomplished  using  TEM2CSV,  a  program  specifically 
developed  for  this  purpose.  TEM2CSV  subtracted  the  site  background  from  the  data  point  using 
a  background  file  specified  by  the  user,  converted  the  points  from  the  geographic  coordinate 
system  used  for  collection  to  the  UTM  Zone  ION  coordinate  system  used  for  processing,  and 
exported  the  resulting  data  to  a  .csv  file  that  could  be  imported  into  The  UX- Analyze  package  in 
Geosoft’s  Oasis  montaj  software.  The  exported  .csv  file  name  contained  both  the  collection  ID 
and  the  Target  ID  (e.g.,  Static00001_2621).  Preprocessing  was  typically  completed  in  batches  of 
half  days,  with  the  day  split  to  account  for  differing  background  data.  Background  files  were 
usually  collected  in  the  mornings  and  afternoons  in  a  geophysically  quiet  location  adjacent  to  the 
IVS  before  the  collection  of  IVS  data.  Unless  there  appeared  to  be  a  problem  with  a  background 
file,  data  collected  before  noon  were  typically  corrected  using  the  morning  background  file;  data 
collected  after  noon  were  corrected  with  the  afternoon  background  file. 

6.2  PARAMETER  ESTIMATION 

All  MetalMapper  data  points  were  inverted  using  UX-Analyze  to  determine  a  number  of 
modeled  parameters  for  each  target.  These  parameters  included  the  location,  size,  and 
orientation  of  the  source  object;  the  polarizability  of  each  axis  of  the  object;  and  information 
regarding  the  quality  of  the  data  and  the  relative  match  between  the  inverted  data  and  the 
expected  model. 

All  target  inversion  was  initially  performed  using  the  UX-Analyze  batch  processing  mode  with 
the  multiple  object  solver  enabled.  Targets  for  which  multiple  objects  were  identified  using  the 
multiple  object  solver  were  reinverted  using  the  batch  processor  without  the  multiple  object 
solver  enabled.  In  these  cases,  the  single  object  and  multiple  object  results  were  compared  to 
determine  which  method  returned  a  result  more  indicative  of  TOI.  Although  the  multiple  object 
result  may  have  approximated  the  expected  model  to  a  higher  degree,  the  result  more  indicative 
of  potential  TOI  was  used  for  target  ranking  to  be  conservative. 
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6.3  CLASSIFIER  AND  TRAINING 


6.3.1  Confidence  Metrics 

The  polarization  curves  developed  for  each  target,  including  any  single-object-only  results  and 
secondary  multiple-object  results,  were  compared  to  a  library  of  known  polarization  curves 
compiled  using  test  stand  data  and  test  pit  data  from  Camp  Beale.  The  items  in  the  Camp  Beale 
comparison  library  were  limited  to  the  TOI  expected  at  the  Camp  Beale  site:  37mm,  75mm,  and 
105mm  projectiles;  60mm  and  81mm  mortars;  and  small  ISOs.  Examples  of  various  types  of 
these  items  were  used  (e.g.,  four  different  versions  of  37mm  projectiles,  three  types  of  105mm 
projectile),  but  items  not  expected  at  the  site,  such  as  hand  grenades  and  rockets,  were  not 
included.  All  initial  comparisons  between  the  measured  targets  and  the  library  data  were  also 
performed  using  an  equal  weight  for  all  three  primary  polarizabilities  (size:  1,  shape  1:  1,  shape 
2:  1).  The  classification  results  for  each  target  were  then  examined  by  the  data  processor. 

The  first  examination  of  the  classification  results  was  performed  to  determine  the  usability  of 
each  result.  The  processor  either  determined  that  the  results  were  usable  as  they  were  or  made  a 
note  in  the  target  database  in  Geosoft  that  further  processing  was  necessary.  Results  were 
deemed  usable  if  the  reviewer  identified  three  reasonable-looking  polarization  curves  or  if  a 
curve  for  the  primary  axis  of  polarization  ((31)  could  not  be  identified.  In  these  cases,  the  target 
was  either  left  for  ranking  according  to  the  decision  statistic  developed  for  the  project  (Section 
4.0)  or,  for  those  targets  without  an  identifiable  (3 1  curve,  classified  as  Can’t  Analyze.  While  the 
data  for  Can’t  Analyze  targets  was  not  usable  for  classification  purposes  because  UX-Analyze 
cannot  effectively  compare  a  target  without  a  primary  polarizability  to  the  library  data,  the  result 
for  that  target  was  considered  “usable”  in  that  no  further  analysis  would  be  performed  on  that 
target. 

Targets  with  results  not  necessarily  deemed  usable  on  the  first  pass  included  those  for  which  one 
or  more  non-(31  curves  appeared  to  be  poor  data  for  any  reason  or  targets  that  appeared  to  be 
“ordnance- like”  but  did  not  have  a  particularly  good  match  to  any  of  the  library  objects. 
Ordnance-like  was  defined  as  an  object  with  relatively  equal  (i.e.,  symmetric)  secondary  axes  of 
polarizability  ([32  and  (33)  for  which  the  magnitude  of  [3 1  was  not  less  than  (32  and  [33.  It  was 
considered  possible  that  targets  with  these  characteristics  were  examples  of  ordnance  not 
expected  at  the  site  and,  therefore,  not  in  the  comparison  library.  Figure  6-1  shows  examples  of 
the  types  of  targets  flagged  during  the  first  examination  of  the  data:  one  with  poor  results  for 
multiple  polarization  curves,  and  one  that  appears  symmetric  but  with  a  poor  match  to  any 
library  object. 
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Figure  6-1:  Targets  Flagged  for  Additional  Processing 

A:  p2  and  P3  curves  appear  to  be  poor  data  B:  Target  is  symmetric,  but  poor  metric 

37mm  90  deg,  10cm  37mm  type  C  -  vertical,  nose  up 

Metric=0.9867  Metric=0.4980 


Targets  with  one  or  more  apparently  poor  P2  or  P3  curves,  as  shown  in  Figure  1A,  were 
recompared  to  the  library  data  with  the  poor  curves  removed  from  the  comparison.  This  was 
accomplished  by  changing  the  comparison  weight  for  the  poor-quality  data  to  0  (size:  1;  shape  1: 
1;  shape  2:  0;  or  size:  1,  shape  1:  0,  shape  2:  0).  In  these  cases,  the  revised,  pi/  p2-  or  pi-only 
based  confidence  metrics  were  used  when  calculating  the  decision  statistic  used  to  rank  each 
target. 


6.3.2  Training  Data 

The  training  data  request  for  Camp  Beale  was  composed  of  targets  flagged  as  ordnance-like,  as 
described  above.  Various  examples  of  these  items  were  added  to  a  separate  target  library  and 
compared  to  the  other  symmetric  objects,  with  the  end  result  that  five  items  added  to  the  library 
fit  all  of  the  others  with  a  confidence  of  0.75  or  higher.  Five  examples  of  each  of  four  of  the 
library  items  (20  total)  were  requested  as  training  data.  The  fifth  object  was  so  similar  to  the  rest 
of  the  examples  from  the  site  that  only  one  example  (BE-2528)  was  requested. 

Out  of  the  requested  targets,  only  BE-2528,  a  native  fuze,  was  identified  as  TOI.  This  target  was 
added  to  the  full  Beale  comparison  library,  and  the  full  list  of  targets  was  compared  to  the  library 
a  second  time  to  determine  final  confidence  metrics  for  each. 

6.3.3  Decision  Statistic 

Classification  for  the  Camp  Beale  project  was  accomplished  using  the  confidence  metrics 
generated  for  each  target  during  the  comparison  to  the  library  data.  The  metric  calculated  for  all 
three  curves  was  used  unless  poor  curves  were  identified  by  the  analyst,  as  described  in  Section 
6.3.1.  In  that  event,  the  metric  calculated  for  the  pl/p2-  or  pi-only  matches  was  used.  The 
decision  statistic  used  for  the  project  was  simply  the  final  confidence  metric  selected  for  each 
target  (pl/p2/p3,  or  pl/p2-  or  pi-only).  No  additional  weight  was  given  to  targets  that  had  three 
usable  curves  rather  than  one-  or  two-only  because  there  were  relatively  few  targets  for  which 
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less  than  three  curves  were  used.  Targets  were  ranked  based  on  decreasing  confidence  that  they 
were  TOI.  The  final  dig  list  was  completed  in  three  stages,  as  described  below 

6.4  DATAPRODUCTS 


6.4.1  Dig  List  Stage  1 

The  Stage  1  dig  list  was  submitted  with  the  following  parameters: 

•  Training  Data:  21  items  selected  as  described  in  Section  6.3.2 

•  Can’t  Analyze:  34  items  without  usable  [31  curves 

•  Likely  TOI  (Category  1):  Decision  statistic  greater  than  0.700 

•  Can’t  Decide  (Category  2):  Decision  statistic  between  0.600  and  0.700 

•  Likely  Clutter  (Category  3):  Decision  statistic  less  than  0.600 

The  stop-dig  threshold  was  set  at  a  decision  statistic  of  0.650  (mid  Category  2),  with  the  Training 
Data  and  Can’t  Analyze  targets  also  considered  digs.  The  Stage  1  dig  list  was  only  compared  to 
the  project  seed  items  identified  as  “QC  seeds.”  This  comparison  identified  two  seeds  that  would 
have  gone  un-dug  based  on  the  Stage  1  list:  BE-2217  and  BE-2277.  BE-2217  was  an  ISO  buried 
at  14cm  with  a  decision  statistic  of  0.638.  Because  this  target  was  one  of  the  un-dug  targets 
within  Category  2,  the  decision  was  made  to  extend  the  dig  threshold  to  the  bottom  of  Category  2 
(0.600)  for  the  next  Stage  of  the  dig  list. 

BE-2277  was  a  37mm  projectile  buried  at  15cm  in  the  vicinity  of  four  pieces  of  barbed  wire 
(Figure  6-2A).  The  decision  statistic  for  this  item  was  0.270,  which  put  it  relatively  close  to  the 
bottom  of  the  dig  list.  The  data  point  for  this  target  was  collected  within  the  25cm  of  the  pick 
location,  the  modeled  location  of  the  target  was  within  10cm  of  the  collection  location,  and  there 
were  no  obvious  signs  of  poor  data  (e.g.,  high  noise  or  poor  signal  strength).  For  this  target,  the 
multiple  object  solver  simply  could  not  separate  the  37mm  response  from  the  barbed  wire 
responses,  resulting  in  a  poor  match  to  any  munition  (Figure  6-2B).  The  nondetection  of  this 
item  led  to  a  reanalysis  of  the  can’t  analyze  category,  although  it  was  finally  decided  that  there 
was  no  reason  to  believe  that  the  data  for  this  target  were  poor. 

6.4.2  Dig  List  Stage  2 

The  Stage  2  dig  list  was  submitted  with  the  following  parameters: 

•  Training  Data:  23  items,  including  QC  Seeds  BE-2217  and  BE-2277 

•  Can’t  Analyze:  45  items  without  usable  [31  curves 

•  Likely  TOI  (Category  1):  Decision  statistic  greater  than  0.650 

•  Can’t  Decide  (Category  2):  Decision  statistic  between  0.650  and  0.575 

•  Likely  Clutter  (Category  3):  Decision  statistic  less  than  0.575 
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Figure  6-2:  Target  BE-2277 

A:  Recovered  Items 
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B:  Response  Curves 
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Modifications  between  the  Stage  1  dig  list  and  the  Stage  2  dig  list  included  the  addition  of  the 
two  missed  QC  seeds  to  the  Training  Data  set,  the  addition  of  11  targets  to  the  Can’t  Analyze 
list,  and  the  changing  of  category  break  points  to  lower  decision  statistics  based  on  the 
nondetection  of  BE-2217  with  the  Stage  1  dig  list.  The  Stage  2  dig  list  was  considered  an  actual 
dig  list,  and  the  full  set  of  intrusive  results  for  the  targets  marked  as  “dig”  were  returned 
following  the  submittal.  For  this  list,  the  stop-dig  point  was  set  at  a  decision  statistic  of  0.600. 

The  dig  results  indicated  that  the  lowest-ranked  TOI  was  BE-2019,  which  had  a  decision  statistic 
of  0.647.  The  final  97  investigated  targets  were  all  non-TOI,  so  there  appeared  to  be  no  reason  to 
modify  the  dig  threshold  further  for  the  final,  Stage  3  dig  list. 

6.4.3  Dig  List  Stage  3 

The  final  dig  list  was  submitted  with  the  following  parameters: 

•  Training  Data:  23  items,  including  QC  Seeds  BE-2217  and  BE-2277 

•  Can’t  Analyze:  45  items  without  usable  |]1  curves 

•  Likely  TOI  (Category  1):  Decision  statistic  greater  than  0.600  -  Dig 

•  Likely  Clutter  (Category  3):  Decision  statistic  less  than  0.600  -  Do  not  dig 

The  only  change  between  the  Stage  2  and  Stage  3  dig  lists  was  the  elimination  of  Category  2. 
Investigated  Can’t  Decide  targets  from  Stage  2  were  changed  to  Category  1,  and  targets  with 
decision  metrics  between  0.600  and  0.575  (i.e.,  un-dug  Category  2)  were  changed  to  Category  3. 
The  final  list  included  422  digs  out  of  the  1,470  total  targets.  Sixty-eight  of  these  were  either 
targets  used  as  Training  Data  or  Can’t  Analyze  anomalies,  with  the  remainder  ranked  according 
to  the  decision  statistic.  The  stop-dig  threshold  was  set  at  rank  number  354. 


29 


7.0  PERFORMANCE  ASSESSMENT 


7.1  OBJECTIVE:  COMPLETE  COVERAGE  OF  THE  DEMONSTRATION  SITE 

The  reliability  of  the  survey  data  depends  on  the  extent  of  coverage  of  the  site.  This  objective 
concerns  the  ability  to  completely  survey  the  site  and  obtain  valid  data. 

7.1.1  Metric 

The  metrics  for  this  objective  were  the  down-line  distance  between  adjacent  data  points  and  the 
maximum  coverage  gap  in  the  final,  mapped  EM61-MK2  data. 

7.1.2  Data  Requirements 

QC  tools  in  Geosoft’s  Oasis  montaj  software  were  used  to  check  the  percentage  of  points  more 
than  25cm  from  each  other  and  the  total  area  of  data  gaps  in  each  surveyed  grid. 

7.1.3  Success  Criteria 

This  objective  was  considered  to  have  been  met  if  at  least  98%  of  the  mapped  data  points  were 
within  25cm  down  line  of  each  other  and  if  at  least  98%  of  the  site  was  mapped  at  the  proposed 
line  spacing  of  50cm. 

7.2  OBJECTIVE:  REPEATABILITY  OF  IVS  MEASUREMENTS 

The  reliability  of  the  survey  data  also  depends  on  the  proper  functioning  of  the  survey 
equipment.  This  objective  concerns  the  twice-daily  confirmation  of  sensor  system  performance. 

7.2.1  Metric 

The  metrics  for  this  objective  were  the  amplitude  and  down-track  position  of  the  maxima 
obtained  from  each  of  the  twice-daily  surveys  of  the  instrument  verification  strip. 

7.2.2  Data  Requirements 

Daily  IVS  survey  data  were  used  to  judge  this  objective. 

7.2.3  Success  Criteria 

This  objective  is  considered  met  if  the  measured  amplitude  for  each  object  in  the  IVS  was  within 
25%  of  the  expected  response  and  if  the  position  of  the  anomaly  was  within  25cm  of  the  known 
location. 

7.3  OBJECTIVE:  DETECTION  OF  ALL  TARGETS  OF  INTEREST 

Quality  data  should  lead  to  a  high  probability  of  detecting  the  TOI  at  the  site. 

7.3.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  seed  items  detected  using  the  specified 
anomaly  selection  threshold. 
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7.3.2  Data  Requirements 

The  target  list  developed  using  the  EM61-MK2  data  was  compared  to  the  known  locations  of  the 
items  seeded  during  the  project  to  judge  this  objective. 

7.3.3  Success  Criteria 

The  objective  is  considered  met  if  100%  of  the  seeded  items  were  detected. 

7.4  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 

The  submitted  dig  list  was  compared  to  ground  truth  data  from  Camp  Beale  by  the  IDA.  The 
IDA  compared  the  results  against  two  sets  of  ground  truth  results.  The  only  difference  between 
the  ground  truth  sets  was  the  way  in  which  32  fuzes  found  at  the  site  were  categorized.  These 
fuzes  were  native  to  the  site  (i.e.,  not  seeds)  and  were  deemed  nonhazardous  MD  when  they  were 
recovered  by  the  dig  teams.  However,  they  were  quite  distinctive  and  were  UXO-related,  and 
there  was  a  fairly  large  number  of  them  at  the  site.  Therefore,  they  were  identified  alternately  as 
UXO  and  MD  by  the  IDA  in  the  two  sets  of  ground  truth  results. 

The  results  of  the  dig  list  comparisons  to  the  two  ground  truth  lists  were  judged  according  to 
performance  objectives  identified  for  the  project  in  the  Camp  Beale  demonstration  (Parsons, 
2011).  Table  3-1  contains  the  performance  objectives  and  identifies  the  criteria  by  which  they 
were  judged.  Results  for  both  sets  of  ground  truth  were  compared  to  the  first  three:  the  correct 
classification  of  TOI,  non-TOI,  and  the  specification  of  the  dig  threshold.  Because  there  was 
only  one  set  of  can’t  analyze  anomalies  and  the  location  of  recovered  items  was  identical 
between  ground  truth  sets,  there  was  only  one  result  for  the  objectives  regarding  the  number  of 
can’t  analyze  anomalies  and  the  offsets  between  recovered  items  and  identified  locations.  The 
results  of  the  submitted  dig  list  with  respect  to  each  project  objective  and  each  set  of  ground  truth 
data,  if  applicable,  are  detailed  below. 

Some  of  the  1,470  targets  for  which  data  were  collected  at  Camp  Beale  ended  up  being  multiple 
picks  on  the  same  source,  so  only  1,441  digs  were  performed  during  the  project.  For  the  fuzes  as 
clutter  ground  truth  set,  131  of  the  1,441  digs  were  classified  as  TOI,  leaving  1,310  true 
negatives  in  this  data  set.  It  also  reduced  the  number  of  items  in  the  dig  list  that  needed  intrusive 
investigation  from  422  to  415.  Figure  7-1  shows  the  receiver  operating  characteristic  (ROC) 
curve  for  the  fuzes  as  clutter  ground  truth  data.  As  indicated  in  the  figure,  130  of  the  131  TOI 
(99.2  percent)  at  the  site  were  correctly  identified  as  targets  that  should  be  dug.  The  one  item 
missed,  BE-1965,  was  a  small  ISO  with  a  decision  metric  of  0.567.  The  performance  objective 
success  criterion  was  the  correct  classification  of  all  TOI,  so  the  misidentification  of  BE-1965  is 
a  failure  with  regard  to  the  objectives. 
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Figure  7-1:  ROC  Curve  for  Fuzes  as  Clutter 
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The  lack  of  inclusion  of  this  target  in  the  dig  category  was  based  on  a  3-curve  comparison 
confidence  metric  of  0.547,  which  was  below  the  0.600  threshold  for  Category  1.  Further 
investigation  of  the  target  indicated  that  the  data  had  a  relatively  low  signal  (signal  amplitude  of 
13.97)  and  some  noise  (beta  noise  value  of  0.11).  Although  neither  of  these  values  is  extremely 
low  or  high,  respectively,  the  analyst  did  indicate  that  the  P2  curve  appeared  to  be  unusable 
(Figure  7-2).  Therefore,  the  confidence  metric  used  for  ranking  this  target  was  the  (3 1  /(32-only  of 
0.567  rather  than  the  0.297  indicated  in  the  figure.  Upon  reinspection,  it  still  does  not  appear 
that  the  pi  curve  is  poor  enough  to  consider  this  a  Can’t  Analyze  target,  although  the  P3  curve 
could  be  considered  unusable.  Using  the  pl-only  result  for  this  target  would  have  resulted  in  a 
confidence  metric  of  0.892  and  classification  as  a  target  that  should  have  been  dug. 
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Figure  7-2:  Polarization  Curves  for  Target  BE-1965 
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For  the  fuzes  as  UXO  ground  truth  set,  164  of  the  1,441  digs  were  classified  as  TOI,  leaving 
1,277  true  negatives  in  this  data  set.  Figure  7-3  shows  the  ROC  curve  for  the  fuzes  as  clutter 
ground  truth  data.  One  hundred  and  fifty-seven  of  the  164  TOI  (95.8  percent)  in  this  data  set 
were  correctly  identified  as  targets  that  should  be  dug.  However,  in  addition  to  BE-1965  (the 
ISO  missed  in  the  fuzes  as  clutter  ground  truth),  six  native  fuzes  were  incorrectly  identified  as 
clutter  in  this  ground  truth  set. 
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Figure  7-3:  ROC  Curve  for  Fuzes  as  UXO 
_fuzesAsUXO 
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While  27  native  fuzes  were  identified  correctly  following  the  addition  of  training  item  BE-2528 
to  the  library,  the  six  missed  were  different  types  than  the  ones  characterized  by  BE-2528.  The 
missed  versions  were  generally  smaller  and  appear  to  be  different  metallically,  as  shown  in 
Figure  7-4.  The  missed  fuzes  generally  agreed  with  each  other  fairly  well  geophysically  in  that 
the  addition  of  two,  BE-2173  and  BE-2225,  to  the  comparison  library  generated  a  match  with  the 
others  with  a  confidence  of  0.750  or  higher.  While  the  BE-2528-type  fuzes  were  numerous  and 
distinctive  enough  to  stand  out  to  the  analyst  as  a  potential  unknown  type  of  UXO,  the  others  did 
not.  It  is  possible  that  the  use  of  a  parameter  space  plot  constructed  using  decay  vs.  size  may 
have  led  to  the  detection  of  these  items,  but  is  not  a  certainty.  Therefore,  it  appears  that  knowing 
these  types  of  fuzes  were  present  and  adding  examples  to  the  comparison  library  would  have 
been  the  only  way  to  guarantee  their  detection  using  library  matching  classification  techniques. 

7.5  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

The  submitted  dig  list  correctly  identified  78.2  percent  (1,024  of  1,310)  of  the  clutter  as  clutter  in 
the  fuzes  as  clutter  ground  truth  comparison  and  79.8  percent  (1,020  of  1,277)  of  the  clutter  as 
clutter  in  the  fuzes  as  UXO  ground  truth  comparison.  Both  versions  are  well  above  the 
performance  objective  of  reducing  the  number  of  false  alarms  by  greater  than  50  percent. 
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Figure  7-4:  Native  Fuze  Types 

A:  BE-2528  B:  BE-2225 


7.6  OBJECTIVE:  CORRECT  SPECIFICATION  OF  NO-DIG  THRESHOLD 

Because  items  were  missed  in  the  comparison  of  the  dig  list  to  both  ground  truth  sets,  the  no-dig 
threshold  on  the  submitted  dig  list  was  set  incorrectly.  In  the  case  of  the  fuzes  as  clutter  ground 
truth  set,  the  last  TOI  was  a  small  ISO  that  had  a  decision  metric  of  0.567.  It  was  47  targets 
below  the  stop-dig  threshold  at  0.600.  If  the  dig  threshold  had  been  correctly  set  at  the  last  TOI 
item,  74.7  percent  of  the  clutter  digs  would  have  been  eliminated. 

The  lowest-ranked  TOI  in  the  fuzes  as  UXO  ground  truth  set  was  BE-2225,  which  was  at  dig 
1,017,  significantly  below  the  dig  threshold.  If  the  dig  threshold  had  been  correctly  set  at  the  last 
TOI  item,  only  33.2  percent  of  the  clutter  digs  would  have  been  eliminated. 

The  no-dig  threshold  in  the  only  dig  list  submitted  was  actually  relatively  low  with  regard  to  the 
decision  statistic  (0.600  out  of  a  possible  1.000)  and  was  based  on  training  and  QC  seed 
information  that  suggested  a  buffer  of  approximately  0.040  between  the  lowest  reasonably 
ranked  QC  seed  item  (decision  metric  of  0.638)  and  the  threshold.  One  of  the  QC  seeds,  BE- 
2277,  a  37mm  buried  in  the  midst  of  a  number  of  pieces  of  barbed  wire,  had  a  decision  statistic 
of  0.270  based  on  a  set  of  polarization  curves  that  looked  nothing  like  ordnance.  Because  the 
curves  appeared  to  be  based  more  on  the  barbed  wire  results  rather  than  the  37mm,  it  was  not 
considered  practical  to  either  add  this  item  to  the  library  or  to  lower  the  dig  threshold  in  order  to 
detect  it.  Therefore,  the  dig  threshold  set  for  the  site  is  still  considered  appropriate;  and  the  TOI 
detection  failures  for  both  sets  of  ground  truth  are  due  to  the  analytical  failures  described  in 
Section  7.4  rather  than  an  incorrectly  located  dig/no-dig  threshold. 

7.7  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE  ANALYZED 

A  total  of  45  targets  were  classified  as  Can’t  Analyze  on  the  single  dig  list  submitted.  This 
corresponds  to  slightly  more  than  the  2  percent  limit  specified  in  the  performance  objectives.  As 
discussed  in  Section  3.3,  all  targets  without  a  usable  pi  curve  were  classified  as  Can’t  Analyze  in 
the  Camp  Beale  data  set.  It  is  expected  that  additional  data  examination,  including  an  analysis  of 
the  signal  amplitude  and  offset  between  the  picked  and  collected  point  locations,  is  likely  to 
reduce  the  number  of  Can’t  Analyze  targets  required.  Using  this  strategy,  potential  Can’t 
Analyze  targets  with  low  signal  amplitude  collected  where  they  should  have  been  collected  (i.e., 
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likely  indicative  of  poor  EM61  data  for  the  original  target)  would  be  considered  good 
MetalMapper  data  despite  poor  pi  curves. 

7.8  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

The  target  parameters  estimated  in  this  case  were  the  X,  Y,  and  relative  Z  (depth)  coordinates  of 
the  targets.  Because  the  goal  with  this  objective  is  to  direct  the  dig  teams  to  the  correct  locations 
for  TOI,  the  comparison  of  estimated  coordinates  to  actual  coordinates  was  only  performed  for 
TOI  and  for  those  targets  marked  as  digs  in  the  ranked  dig  list.  In  the  TOI  version,  the  fuzes  as 
UXO  ground  truth  set  was  used  for  comparison. 

The  success  criteria  for  this  performance  objective  were  X,  Y  offsets  for  which  one  standard 
deviation  of  the  dataset  was  less  than  15cm  and  one  standard  deviation  of  the  depth  offset  was 
less  than  10cm.  The  performance  objective  was  passed  for  the  TOI  comparison,  with  calculated 
standard  deviations  of  9.6cm  for  the  horizontal  offset  and  5.0cm  for  the  vertical  offset.  The 
results  for  all  of  the  targets  marked  as  digs  were  slightly  above  the  criteria  for  the  horizontal 
offset,  with  calculated  standard  deviation  of  19.3cm.  The  depth  calculation  was  within  the 
criteria  with  a  standard  deviation  of  9.1cm. 

In  the  comparisons,  the  modeled  location  of  the  target  was  the  fit  X,  Y,  and  Z  coordinates,  while 
the  actual  location  was  defined  as  the  closest  measured  location  to  the  fit  location.  This  only 
mattered  in  the  case  of  targets  for  which  multiple  objects  were  recovered  from  the  vicinity  of  the 
picked  location.  However,  much  of  the  discrepancy  seen  in  the  X,  Y  locations  for  all  of  the 
“dig”  targets  is  likely  due  to  multiple-object  locations  being  modeled  as  single  objects  during 
inversion.  While  the  multiple  object  solver  option  in  UX- Analyze  was  employed  for  inversion,  it 
does  not  appear  to  have  been  successful  in  every  case.  The  notable  example  is  BE-2277,  the 
37mm  QC  seed  buried  in  the  vicinity  of  a  number  of  pieces  of  barbed  wire  that  was  not  identified 
as  multiple  sources  and  looked  nothing  like  UXO.  There  does  not  seem  to  be  a  ready  solution  to 
the  slight  discrepancy  between  the  performance  objective  and  the  results  for  the  “dig”  targets, 
although  the  results  for  the  TOI  were  positive. 


8.0  COST  ASSESSMENT 

The  cost  assessment  was  split  into  two  groups:  MetalMapper  costs  and  conventional  intrusive 
costs.  The  MetalMapper  costs  include  instruments,  surveying,  seeding,  and  analysis  costs;  the 
conventional  intrusive  costs  include  surface  sweep,  data  collection,  and  intrusive  costs. 

8.1  COST  MODEL 

The  cost  model  for  the  Former  Camp  Beale  demonstration  includes  the  total  cost  of  the  project 
and  potential  savings  from  the  classification  process.  The  total  cost  includes  the  seeding 
operation,  MetalMapper  operations,  processing,  and  intrusive  operation.  Estimates  for  each 
operation  are  listed  in  Table  9-1. 

Although  the  MetalMapper  and  tractor  used  for  data  collection  at  Camp  Beale  were  funded  by 
ESTCP,  standard  rental  costs  and  prep  fees  were  used  to  determine  the  cost  for  the  Camp  Beale 
project  had  Parsons  rented  these  items  for  the  duration  of  the  project.  Weekly  costs  for 
equipment  on  a  typical  project  were  calculated  using  only  items  that  would  be  required  each 
week  and  not  items  like  mobilization,  prep,  or  shipping  fees  that  would  be  one-time  items  on 
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most  projects.  Survey  and  analysis  costs  were  tracked  using  a  task-specific  number  in  Parsons 
project  controls  system. 

The  final  result  of  the  classification  process  provides  an  alternative  approach  to  the  final  costs  of 
the  intrusive  operations.  The  analysis  compares  costs  of  using  MetalMapper  vs.  digging  all 
anomalies  classified  as  no-digs.  At  the  former  Camp  Beale,  classification  eliminated  78%  of 
digs  at  a  cost  of  $59  per  target. 

The  overall  cost  of  excavating  the  1,491  MetalMapper  anomalies  was  $188,549  ($  126/anomaly). 
With  a  reduction  of  1,020  excavations,  this  would  represent  a  cost  savings  of  $75,480.  However, 
471  anomalies  would  have  required  MetalMapper  data  collection  and  processing  at  an  estimated 
cost  of  $24,492  ($59  x  471).  Therefore  the  overall  cost  savings  would  have  represented  $50,988. 


Table  9-1:  Details  of  the  Costs  That  Will  Be  Tracked. 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

Seeding/EM61  Survey  Costs 

Seed  Emplacement/initial 

Costs  for  surface  sweep,  seed  emplacement, 

$114,287 

set  up 

surveying  seeds 

Survey  costs 

The  survey  costs  will  include  the  time  spent 
in  the  field  collecting  and  recording  data 

$75,055 

MetalMapper  Survey  Costs 

MetalMapper  Rental  ($500/day;  21  days) 

$10,500 

MetalMapper  Prep  Fee  (project) 

$1,000 

MetalMapper  Shipping  (project) 

$2,700 

Tractor  Rental  (project) 

$1,440 

Tractor  Mob/Demob  (project) 

$100 

Instrument  Costs 

$2,400 

RTK  GPS  Cost  ($800/week;  3  weeks) 

Shipping  (RTK  GPS,  etc;  project) 

$324 

Total 

$18,464 

Per  Target 

$12.38 
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Table  9-1:  Details  of  the  Costs  That  Will  Be  Tracked  (Continued). 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

MetalMapper  Survey  Costs  (Continued) 

Field-related  labor  (2  geophysicists,  UXO 
Technician  II),  equipment  set-up,  test  pit 
data  collection,  cued  data  collection, 

$56,869 

Survey  Costs 

preprocessing,  initial  target  inversion  for  QC 
checks,  non-equipment  direct  costs  (per 
diem,  hotel,  fuel,  and  etc.) 

Per  Target 

$37.91 

Analysis  Costs 

All  processing  and  analysis  performed 
following  the  completion  of  field  activities 

$13,566 

Per  Target 

$9.04 

Intrusive  Costs 

All  costs  related  to  the  intrusive 

$271,389 

Investigations 

investigation 

Cost  per  anomaly  to  intrusively  investigate 

$126 

8.2  COST  DRIVERS 

Based  on  the  factors  described  above,  the  total  per  target  cost  for  the  MetalMapper-related  work 
Camp  Beale  project  was  $59.33.  Although  backed  by  the  actual  costs  for  the  project,  Parsons 
considers  this  a  fairly  significant  overestimation  of  the  actual  costs  that  would  be  necessary  for 
future  projects  based  on  the  following  factors  specific  to  Camp  Beale: 

•  Training  required  for  field  team  that  was  completely  unfamiliar  with  the  MetalMapper; 

•  Unfamiliarity  also  led  to  a  significant  number  of  re-shots  during  the  first  week  of  the 
project;  and 

•  Numerous  software  crashes  related  to  some  combination  of  the  GPS  unit,  the  IMU, 
Windows  7,  the  data  acquisition  program,  and/or  the  touch  screen  on  the  MetalMapper 
resulted  in  a  significant  number  of  software  crashes. 

When  the  MetalMapper  worked  as  planned,  the  field  team  generally  had  no  problem  collecting 
data  at  a  rate  of  greater  than  200  points  per  day.  However,  the  issues  discussed  above  resulted  in 
an  average  production  rate  of  only  117  points  per  day.  The  software  crashes  were,  by  far,  the 
largest  cause  of  the  relatively  low  production,  with  significant  portions  of  some  days  spent  trying 
to  fix  the  problem.  The  specific  cause  of  the  crashes  is  still  unknown,  but  it  should  be  reasonable 
to  assume  that  the  root  cause  can  be  fixed  before  the  MetalMapper  is  used  on  a  project  with  a 
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significant  number  of  targets.  Once  fixed,  a  production  rate  of  twice  the  one  achieved  at  Camp 
Beale  is  not  out  of  the  question. 

For  the  intrusive  investigation  phase  of  the  Camp  Beale,  Parsons  investigated  2,  143  anomalies 
in  23  intrusive  days.  The  average  anomalies  intrusively  investigated  per  day  was  93  anomalies 
which  exceeded  the  estimated  80  anomalies  per  day  planned  for  the  project.  The  most  common 
item  excavated  at  Camp  Beale  was  frag.  Efficiency  among  the  intrusive  teams  was  the  main 
factor  in  the  production  or  lack  of  production  throughout  the  project. 

•  The  predetermined  path  of  processing  a  group  of  anomalies  which  require  the  least 
amount  of  movements  for  the  intrusive  teams  with  all  their  equipment  was  the  single 
most  important  factor  in  production. 

•  Other  factors  related  to  production  included  multiple  contact  anomalies  which  were 
frequent  based  on  the  low  EM61-MK2  channel  two  threshold  for  QC,  identifying  frag 
and  barbed  wire  pits  early  potentially  saved  intrusive  time. 

•  Moving  heavy  equipment  throughout  the  site  was  very  time  consuming  on  the  very 
uneven  terrain  at  Camp  Beale  but  proper  planning  minimized  this  need. 

•  Pinpointing  small  anomalies  quickly  with  the  EM61-MK2  minimizes  standby  time  of 
the  actual  digging;  small  pieces  of  frag  were  difficult  to  locate  with  the  wide  EM61- 
MK2  sensor.  Having  an  experienced  EM61-Mk2  operator  helped  this  process. 

•  The  weather  was  a  big  factor  during  the  summer  months,  areas  with  no  shade  required 
more  breaks  and  water  intake  for  the  intrusive  teams. 


8.3  COST  BENEFIT 

For  a  production  removal  action  project  with  10,000  anomalies  selected  for  investigation,  we 
would  expect  the  MetalMapper  costs  to  be  reduced  to  approximately  $39/anomaly  (based  on  a 
production  rate  of  200/day)  for  data  collection  and  processing.  We  would  also  expect  the 
intrusive  costs  to  be  closer  to  $100/anomaly.  Assuming  a  75  percent  reduction  in  the  number  of 
clutter  items  that  could  be  eliminated  from  intrusive  investigation  would  yield  a  potential  cost 
savings  of  $360,000  based  on  the  following  assumptions: 

•  10,000  anomalies  at  $100/anomaly  for  intrusive  investigation  equals  a  cost  of 
$1,000,000. 

•  Reduction  of  7,500  anomalies  equals  a  reduction  of  $750,000  in  excavation  costs. 

•  MetalMapper  costs  for  analyzing  10,000  anomalies  at  $39/anomaly  equals  a  cost  of 
$390,000. 

•  Total  net  savings  under  this  scenario  equals  $360,000  (36  percent). 


9.0  IMPLEMENTATION  ISSUES 

There  were  a  few  notable  implementation  issues  regarding  the  MetalMapper  survey: 
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•  The  EM3D  software  crashes  discussed  briefly  in  Section  8.2  were  the  largest  issue  faced 
by  the  field  team.  There  were  very  few  days  during  the  field  project  in  which  these 
crashes  dig  not  cause  delays  ranging  from  an  hour  or  so  of  lost  production  to  most  of  the 
day.  Parsons  used  the  MetalMapper  from  Camp  Beale  for  data  collection  during  another 
project  at  Fort  Sill,  Oklahoma,  and  the  crashing  problems  persisted.  This  is  an  issue  that 
needs  to  be  resolved  before  consistent  production  of  200+  points  per  day  can  be  achieved. 

•  The  field  crew  using  the  MetalMapper  at  Camp  Beale  was  completely  new  to  the 
technology.  As  could  be  expected,  production  and  data  quality  was  low  while  the  crew 
familiarized  themselves  with  the  MetalMapper.  Data  quality  issues  were  based  on  the 
number  of  re-shots  required  for  data  collected  early  in  the  project,  when  re-shot 
percentages  were  up  to  approximately  20  percent  of  the  collected  points.  By  the  end  of 
the  project,  re-shot  rates  were  down  to  a  few  targets  per  day. 

•  During  the  collection  and  analysis  of  the  data,  little  consideration  was  given  to  the 
distance  between  the  picked  target  location  from  the  electromagnetic  survey  and  the 
collection  and  fit  locations  of  the  MetalMapper  points  for  the  targets.  Future  analysis 
should  consider  these  offsets,  both  in  the  field  to  ensure  that  MetalMapper  points  are 
collected  within  a  reasonable  distance  of  the  intended  point  and  during  analysis  to  ensure 
that  the  results  generated  for  a  given  point  actually  represent  that  point. 

•  All  targets  with  poor  (31  curves  were  classified  as  can’t  analyze  for  this  project.  It  is 
likely  that  some  of  the  Can’t  Analyze  targets  were  collected  at  EM61  target  locations  that 
may  have  been  generated  as  targets  due  to  EM61  noise  or  duplicate  pick  on  an  anomaly 
better  represented  by  a  different  EM61  target.  In  cases  such  as  these,  an  analysis  of  the 
signal  amplitude  and  EM  target  vs.  MetalMapper  collection  vs.  MetalMapper  fit  locations 
may  have  resulted  in  fewer  Can’t  Analyze  targets. 
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APPENDIX  A 


POINTS  OF  CONTACT 


Appendix  A:  Points  of  Contact 


POINT  OF 
CONTACT 

ORGANIZATION 

Phone 

Fax 

e-mail 

Role  in  Project 

Dr.  Jeff 

Marqusee 

ESTCP  Program  Office 

901  North  Stuart  Street, 

Suite  303 

Arlington,  VA  22203 

703-696-2120  (V) 

703-696-2114  (F) 
jeffrey.marqusee@osd.mil 

Director,  ESTCP 

Dr.  Anne 

Andrews 

ESTCP  Program  Office 

901  North  Stuart  Street, 

Suite  303 

Arlington,  VA  22203 

703-696-3826  (V) 

703-696-2114  (F) 
anne.andrews@osd.mil 

Deputy  Director, 
ESTCP 

Dr.  Herb  Nelson 

ESTCP  Program  Office 

901  North  Stuart  Street, 

Suite  303 

Arlington,  VA  22203 

703-696-8726  (V) 

703-696-2114  (F) 

202-215-4844  (C) 
herbert.nelson@osd.mil 

Program  Manager, 

MR 

Ms.  Katherine 
Kaye 

HydroGeoLogic,  Inc. 

11107  Sunset  Hills  Road, 
Suite  400 

Reston,  VA  20190 

410-884-4447  (V) 
kkaye@hgl.com 

Special  Projects 
Consultant,  MR 

Mr.  Daniel 

Ruedy 

HydroGeoLogic,  Inc. 

11107  Sunset  Hills  Road, 
Suite  400 

Reston,  VA  20190 

703-736-4531 

druedy@hgl.com 

Program  Assistant, 

MR 

Dr.  Shelley 
Cazares 

Institute  for  Defense 

Analyses 

4850  Mark  Center  Drive 
Alexandria,  VA  223 1 1 

703-845-6792  (V) 

703-578-2877  (F) 
scazares@ida.org 

Performance 

Assessment 

Ms.  Jane  Francis 

Wyoming  Dept,  of 
Environmental  Quality 

122  West  25  th  Street 
Herschler  Bldg.,  4-W 
Cheyenne,  WY  82002 

307-777-7092  (V) 
jfranc@wyo.gov 

State  Regulator 

Ms.  Adrienne 
Nunn 

Wyoming  Dept,  of 
Environmental  Quality 

122  West  25  th  Street 
Herschler  Bldg.,  4-W 
Cheyenne,  WY  82002 

307-777-6428  (V) 
anunn@wyo.gov 

State  Regulator 

A-l 


Mr.  Larry 
Sandoval 

Medicine  Bow-Routt  NF 

307-745-2337  (V) 
lwsandoval@fs.fed.us 

Laramie  District 
Ranger 

Mr.  Nate  Haynes 

Medicine  Bow  NF 

Laramie  Ranger  District 

2468  Jackson  St.,  Laramie, 
WY  82070 

307-745-2317 

nhaynes@fs.fed.us 

On  Site  Contact 

Ms.  Laura 
Percifield 

Project  Manager 
Environmental  Remediation 
Branch 

US  ACE  -  Omaha  District 

402-995-2761  (V) 
laura.j.percifield@usace.anny.mil 

USACE 

Representative 

Dr.  Stephen 
Billings 

Sky  Research,  Inc. 

445  Dead  Indian  Memorial 
Road 

Ashland,  OR  97520 

541-552-5185  (V) 

541-488-4606  (F) 
steve.billings@skyresearch.com 

Lead  PI 

Mr.  Heesoo 

Chung 

Sky  Research,  Inc. 

303-868-9309 

Heesoo.Chung@skyresearch.com 

Lead  Data  Collector 

Ms.  Victoria  A. 
Kantsios 

URS  Group,  Inc. 

2450  Crystal  Drive 

Suite  500 

Arlington,  VA  22202 

703-418-3030  (V) 

703-418-3040  (F) 

victoria  kantsios@urscorp.com 

Lead  PI 

Mr.  Darrell  Hall 

URS  Group,  Inc. 

402-334-8181 

darrell  hall@urscorp.com 

Project  Geophysicist 

Dr.  Dean 
Keiswetter 

SAIC 

120  Quade  Drive 

Cary,  NC  27513 

919-677-1560  (V) 

919-678-1508  (F) 
keiswetterd@saic  .com 

Lead  PI 

Mr.  Levi 

Kennedy 

Signal  Innovations  Group, 

Inc. 

1009  Slater  Rd„  Ste.  200 
Durham,  NC  27703 

919-323-3456  (V) 

919-287-2398  (F) 
lkennedy@siginnovations.com 

Lead  PI 

Dr.  Len  Pasion 

Sky  Research,  Inc. 

Suite  1 12A,  2386  East  Mall 
Vancouver,  BC  V6T  1Z3 

541-552-5185  (V) 

541-488-4606  (F) 
len.pasion@skyresearch.com 

Lead  PI 

A-2 


Mr.  Greg  Van 

Parsons 

1700  Broadway,  Suite  900 
Denver,  CO  80290 

303-764-1927  (V) 

303-831-8208  (F) 
Greg.Van@Parsons.com 

Lead  PI 

Mr.  John 

Baptiste 

Parsons 

1700  Broadway,  Suite  900 
Denver,  CO  80290 

303-764-8840 

John.E.Baptiste@parsons.com 

Data  Analyst 

Ms.  Elise  Goggin 

US  ACE,  Huntsville 

256-895-1635 

Elise.M.Goggin@usace.anny.mil 

Data  Analyst 

Mr.  Rick 
Grabowski 

USACE,  Omaha  District 

402-995-2284 

richard.j.grabowski@usace.army.mil 

Data  Analyst 

Mr.  Tim  Deignan 

Shaw  Environmental,  Inc. 

720-554-8273 

timothy .  deignan@shawgrp  .com 

Data  Analyst 

A-3 


